yvvki commited on
Commit
c071d57
·
verified ·
1 Parent(s): 06ec15a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +71 -22
README.md CHANGED
@@ -1,5 +1,13 @@
1
  ---
2
- base_model: []
 
 
 
 
 
 
 
 
3
  library_name: transformers
4
  tags:
5
  - mergekit
@@ -8,44 +16,85 @@ tags:
8
  ---
9
  # Erotophobia-24-v2.0
10
 
 
 
11
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
 
13
- ## Merge Details
14
  ### Merge Method
15
 
16
- This model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using ./Erotophobia-24-v2.0/intermediates/Mind as a base.
 
 
 
 
 
 
17
 
18
  ### Models Merged
19
 
20
  The following models were included in the merge:
21
- * ./Erotophobia-24-v2.0/intermediates/Soul
22
- * ./Erotophobia-24-v2.0/intermediates/Heart
 
 
 
 
 
 
 
23
 
24
  ### Configuration
25
 
26
- The following YAML configuration was used to produce this model:
27
 
28
  ```yaml
29
- base_model: ./Erotophobia-24-v2.0/intermediates/Mind
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  chat_template: auto
 
 
 
 
 
31
  merge_method: della
32
- modules:
33
- default:
34
- slices:
35
- - sources:
36
- - layer_range: [0, 40]
37
- model: ./Erotophobia-24-v2.0/intermediates/Mind
38
- - layer_range: [0, 40]
39
- model: ./Erotophobia-24-v2.0/intermediates/Heart
40
- parameters:
41
- weight: 0.6
42
- - layer_range: [0, 40]
43
- model: ./Erotophobia-24-v2.0/intermediates/Soul
44
- parameters:
45
- weight: 0.4
46
  parameters:
47
  density: 0.7
48
  epsilon: 0.2
49
  lamda: 1.1
50
- tokenizer: {}
51
  ```
 
1
  ---
2
+ base_model:
3
+ - huihui-ai/Arcee-Blitz-abliterated
4
+ - darkc0de/BlackXorDolphTronGOAT
5
+ - PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
6
+ - aixonlab/Eurydice-24b-v3.5
7
+ - Gryphe/Pantheon-RP-1.8-24b-Small-3.1
8
+ - TheDrummer/Cydonia-24B-v2.1
9
+ - ReadyArt/Broken-Tutu-24B
10
+ - ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
11
  library_name: transformers
12
  tags:
13
  - mergekit
 
16
  ---
17
  # Erotophobia-24-v2.0
18
 
19
+ ## Merge Details
20
+
21
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
22
 
 
23
  ### Merge Method
24
 
25
+ This model was merged using multi-merge methods:
26
+
27
+ - **Mind** was merged using the [Arcee Fusion](https://www.arcee.ai/blog/meet-mergekit-v0-1-arcee-fusion-expanded-model-support-multi-gpu-acceleration) merge method using huihui-ai/Arcee-Blitz-abliterated as a base.
28
+ - **Heart** was merged using the [Karcher Mean](https://en.wikipedia.org/wiki/Karcher_mean) merge method.
29
+ - **Soul** was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using TheDrummer/Cydonia-24B-v2.1 as a base.
30
+
31
+ Fially, this model was merged using the [DELLA](https://arxiv.org/abs/2406.11617) merge method using **Mind** as a base.
32
 
33
  ### Models Merged
34
 
35
  The following models were included in the merge:
36
+ * **Mind**
37
+ - darkc0de/BlackXorDolphTronGOAT
38
+ * **Soul**
39
+ - PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
40
+ - aixonlab/Eurydice-24b-v3.5
41
+ - Gryphe/Pantheon-RP-1.8-24b-Small-3.1
42
+ * **Heart**
43
+ - ReadyArt/Broken-Tutu-24B
44
+ - ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
45
 
46
  ### Configuration
47
 
48
+ The following YAML configuration was used to produce this model (using `mergekit-multi`):
49
 
50
  ```yaml
51
+ name: Mind
52
+ merge_method: arcee_fusion
53
+ dtype: bfloat16
54
+ tokenizer:
55
+ source: union
56
+ chat_template: auto
57
+ base_model: huihui-ai/Arcee-Blitz-abliterated
58
+ models:
59
+ - model: darkc0de/BlackXorDolphTronGOAT
60
+ ---
61
+ name: Heart
62
+ merge_method: karcher
63
+ tokenizer:
64
+ source: union
65
+ chat_template: auto
66
+ parameters:
67
+ max_iter: 1000
68
+ models:
69
+ - model: PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
70
+ - model: aixonlab/Eurydice-24b-v3.5
71
+ - model: Gryphe/Pantheon-RP-1.8-24b-Small-3.1
72
+ ---
73
+ name: Soul
74
+ merge_method: model_stock
75
+ tokenizer:
76
+ source: union
77
  chat_template: auto
78
+ base_model: TheDrummer/Cydonia-24B-v2.1
79
+ models:
80
+ - model: ReadyArt/Broken-Tutu-24B
81
+ - model: ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
82
+ ---
83
  merge_method: della
84
+ tokenizer:
85
+ source: union
86
+ chat_template: auto
87
+ base_model: Mind
88
+ models:
89
+ - model: Mind
90
+ - model: Heart
91
+ parameters:
92
+ weight: 0.6
93
+ - model: Soul
94
+ parameters:
95
+ weight: 0.4
 
 
96
  parameters:
97
  density: 0.7
98
  epsilon: 0.2
99
  lamda: 1.1
 
100
  ```