schonsense/70B_SOG_MMSLERPV2

SOG_MMSLERP

Turn the temp down on this one. 0.1 to 0.6.

This is a multi-merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the MULTI-merge method.

Models Merged

The following models were included in the merge:

• schonsense/70B_SOG_unstructed
• nbeerbower/Llama3.1-Gutenberg-Doppel-70B
• flammenai/Llama3.1-Flammades-70B
• flammenai/Mahou-1.5-llama3.1-70B
• D:\mergekit\SOG_MSLERP_MULTI
• D:\mergekit_My_YAMLS\70B_mSlOG_un

Configuration

The following YAML configuration was used to produce this model:

name: flam
merge_method: multislerp

models:

  - model: nbeerbower/Llama3.1-Gutenberg-Doppel-70B
  - model: flammenai/Llama3.1-Flammades-70B
  - model: flammenai/Mahou-1.5-llama3.1-70B
parameters:
  weight: 1

---
name: SOG_MSLERP_MULTI
merge_method: della
models:
  - model: flam 
    parameters:
      density: 0.2
      epsilon: 0.1
      weight: 0.2
     
  - model: "D:\\mergekit\\_My_YAMLS\\70B_mSlOG_un"
    parameters:
      density: 1
      epsilon: 0
      weight: 0.8

base_model: "D:\\mergekit\\_My_YAMLS\\70B_mSlOG_un"
parameters:
  normalize: false
  int8_mask: false
  lambda: 1.0

---
models:

  - model: "D:\\mergekit\\_My_YAMLS\\70B_mSlOG_un"
    parameters:
      weight: 1

  - model: SOG_MSLERP_MULTI
    parameters:
      weight: 1
       

merge_method: nuslerp 
tokenizer_source: "D:\\mergekit\\_My_YAMLS\\70B_mSlOG_un"

dtype: float32

out_dtype: bfloat16