Hugging Face's logo

Retreatcost
/
Ollpheist-12B

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
frankenmerge
conversational
Eval Results
text-generation-inference
Model card Files
xet
Community
Ollpheist-12B / README.md
Retreatcost's picture
Retreatcost
Update README.md
a462093 verified
preview code
|
raw
history blame contribute delete
3.74 kB
metadata
base_model:
  - yamatazen/LorablatedStock-12B
  - yamatazen/EtherealAurora-12B
  - DreadPoor/Irix-12B-Model_Stock
  - yamatazen/BlueLight-12B
library_name: transformers
tags:
  - mergekit
  - merge
  - frankenmerge
license: apache-2.0
model-index:
  - name: Retreatcost/Ollpheist-12B
    results:
      - task:
          type: text-generation
          name: UGI score
        metrics:
          - name: UGI
            type: ugi
            value: 30.77
        source:
          name: UGI Leaderboard
          url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard
      - task:
          type: text-generation
          name: W/10 Score
        metrics:
          - name: W/10
            type: willingness
            value: 6.2
        source:
          name: UGI Leaderboard
          url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard
      - task:
          type: text-generation
          name: NatInt Score
        metrics:
          - name: NatInt
            type: natint
            value: 23.27
        source:
          name: UGI Leaderboard
          url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard
      - task:
          type: text-generation
          name: Writing Score
        metrics:
          - name: Writing
            type: writing
            value: 37.42
        source:
          name: UGI Leaderboard
          url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard
      - task:
          type: text-generation
          name: NSFW score
        metrics:
          - name: NSFW
            type: nsfw
            value: 5.9
        source:
          name: UGI Leaderboard
          url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard
      - task:
          type: text-generation
          name: Dark score
        metrics:
          - name: Dark
            type: dark
            value: 6.3
        source:
          name: UGI Leaderboard
          url: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard

Ollpheist-12B

image/png

Probably a good generalist model.

Merge Details

Merge Method

This model was merged using the Karcher Mean merge method.

Models Merged

The following models were included in the merge:

Reproduction steps

Spoiler warning 
merge_method: passthrough
slices:
- sources: # Best Willingness 
  - model: yamatazen/LorablatedStock-12B
    layer_range: [0, 6]

- sources: # Best NatInt
  - model: yamatazen/EtherealAurora-12B
    layer_range: [6, 34]

- sources: # Best UGI 
  - model: DreadPoor/Irix-12B-Model_Stock
    layer_range: [34, 40]    
dtype: bfloat16

mergekit-yaml ollpheist_base.yml ./Ollpheist-base-12B

merge_method: arcee_fusion
base_model: ./Ollpheist-12B
models:
  - model: ./Ollpheist-12B
  - model: yamatazen/BlueLight-12B # Best Coding 
dtype: bfloat16
tokenizer_source: ./Ollpheist-12B

mergekit-yaml ollpheist_fusion.yml ./Ollpheist-fusion-12B

merge_method: karcher
models:
  - model: ./Ollpheist-base-12B
  - model: ./Ollpheist-fusion-12B
parameters:
  max_iter: 100000
  tol: 1e-9
dtype: bfloat16

mergekit-yaml ollpheist_karcher.yml ./Ollpheist-k-12B

merge_method: karcher
models:
  - model: ./Ollpheist-base-12B
  - model: ./Ollpheist-k-12B
parameters:
  max_iter: 100000
  tol: 1e-9
dtype: bfloat16

mergekit-yaml ollpheist_karcher2.yml ./Ollpheist-12B

Acknowledgments