metadata

language:
  - en
license: apache-2.0
library_name: transformers
tags:
  - merge
  - mergekit
  - lazymergekit
  - ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
  - invisietch/EtherealRainbow-v0.3-8B
base_model:
  - ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
  - invisietch/EtherealRainbow-v0.3-8B
pipeline_tag: text-generation
model-index:
  - name: Llama-3.1-8B-RainbowLight-EtherealMix
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: IFEval (0-Shot)
          type: HuggingFaceH4/ifeval
          args:
            num_few_shot: 0
        metrics:
          - type: inst_level_strict_acc and prompt_level_strict_acc
            value: 49.73
            name: strict accuracy
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: BBH (3-Shot)
          type: BBH
          args:
            num_few_shot: 3
        metrics:
          - type: acc_norm
            value: 31.07
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MATH Lvl 5 (4-Shot)
          type: hendrycks/competition_math
          args:
            num_few_shot: 4
        metrics:
          - type: exact_match
            value: 12.16
            name: exact match
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GPQA (0-shot)
          type: Idavidrein/gpqa
          args:
            num_few_shot: 0
        metrics:
          - type: acc_norm
            value: 4.92
            name: acc_norm
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MuSR (0-shot)
          type: TAUR-Lab/MuSR
          args:
            num_few_shot: 0
        metrics:
          - type: acc_norm
            value: 9.87
            name: acc_norm
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU-PRO (5-shot)
          type: TIGER-Lab/MMLU-Pro
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 29.23
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
          name: Open LLM Leaderboard

ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix

Overview

ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix is a powerful fusion of ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes and invisietch/EtherealRainbow-v0.3-8B, utilizing SLERP (Spherical Linear Interpolation) for optimal blending of embeddings. This merge enhances reasoning, contextual understanding, and creative language generation while retaining ethical alignment and responsiveness.

🔥 Merged Models

ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes - A highly optimized instruction-tuned model, built for nuanced, long-form reasoning.
invisietch/EtherealRainbow-v0.3-8B - A dynamic conversational model with expanded alignment and expressiveness.

⚙️ Merge Configuration

The following YAML configuration defines how these models were fused using SLERP:

# Merge configuration for ZeroXClem-Llama-3.1-8B-RainbowLight-EtherealMix using SLERP

name: ZeroXClem-Llama-3.1-8B-RainbowLight-EtherealMix
slices:
  - sources:
      - model: ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
        layer_range: [0, 32]
      - model: invisietch/EtherealRainbow-v0.3-8B
        layer_range: [0, 32]
merge_method: slerp
base_model: invisietch/EtherealRainbow-v0.3-8B
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

Why SLERP?

Maintains Model Integrity: Ensures a smooth transition between feature spaces of both models.
Preserves Semantic Meaning: Avoids interpolation collapse, keeping token embeddings rich in structure.
Balanced Performance: Retains the best qualities from both parent models.

🚀 Capabilities

🌟 Enhanced Features

Supercharged Instruction Following – More intuitive and context-aware.
Advanced Conversational Flow – Generates human-like responses with coherence.
Creative and Expressive Writing – Ideal for storytelling, summarization, and content generation.
Expanded Knowledge Base – Merging brings broader factual recall and conceptual understanding.
Flexible Alignment – A balance of compliance and open-ended response generation.

📥 Usage Instructions

Transformers

You can use the model via Hugging Face's transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Sample inference
prompt = "What are the implications of artificial intelligence in the future of education?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Ollama

For local execution with Ollama:

ollama run hf.co/ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix

📌 Important Notes

License: Governed by Meta's Llama 3.1 Community License.
Alignment Considerations: Users are responsible for ethical and compliant use.
System Tokens: Follows Llama 3.1 tokenization standards for inference stability.
Quantization: Use FP16 for optimal performance, though Q8 quantized versions may be available.

💜 Special Thanks

Deep gratitude to:

@invisietch for EtherealRainbow-v0.3-8B.
Hugging Face & Open-Source AI Community for their incredible contributions. 🚀💖

🔗 Resources

✨ Merged with precision. Optimized for excellence. Experience RainbowLight EtherealMix today! ✨

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	22.83
IFEval (0-Shot)	49.73
BBH (3-Shot)	31.07
MATH Lvl 5 (4-Shot)	12.16
GPQA (0-shot)	4.92
MuSR (0-shot)	9.87
MMLU-PRO (5-shot)	29.23