R136a1 commited on
Commit
c75c491
·
verified ·
1 Parent(s): cdf5ace

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -19
README.md CHANGED
@@ -6,24 +6,16 @@ tags:
6
  - safetensors
7
  - mixtral
8
  ---
9
- Test model.
10
 
11
- Under testing...
12
 
13
- Recipe:
14
- ```yaml
15
- base_model: /content/InfinityRP
16
- gate_mode: random
17
- dtype: bfloat16 # output dtype (float32, float16, or bfloat16)
18
- ## (optional)
19
- experts_per_token: 2
20
- experts:
21
- - source_model: /content/Aurav2
22
- positive_prompts: []
23
- - source_model: /content/Spice
24
- positive_prompts: []
25
- - source_model: /content/InfinityRP
26
- positive_prompts: []
27
- - source_model: /content/DaCo
28
- positive_prompts: []
29
- ```
 
6
  - safetensors
7
  - mixtral
8
  ---
 
9
 
10
+ I prefer this one instead of v1 since it's a bit more creative and _smart_, understand the story better. This use some different models from the v1 but perform very close to it (I guess since I used the same model for the base?).
11
 
12
+
13
+ Testing done.
14
+
15
+ It performs really well in complex scenario and follows the character card quite well. The char card and previous message can affect a lot to the next reply style.
16
+
17
+ The main idea is instead of _merging_ models to create new model, I try to put these best model into mixtral so it can work together. And the result is good, every model has its uniqueness and strength.
18
+
19
+ Downside? it only support 8k (8192) context length...
20
+
21
+ Alpaca prompting format.