README.md

---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- merge
- mergekit
- lazymergekit
- ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
- invisietch/EtherealRainbow-v0.3-8B
base_model:
- ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
- invisietch/EtherealRainbow-v0.3-8B
pipeline_tag: text-generation
model-index:
- name: Llama-3.1-8B-RainbowLight-EtherealMix
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 49.73
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 31.07
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 12.16
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 4.92
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 9.87
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 29.23
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
      name: Open LLM Leaderboard
---

# ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix

## Overview
**ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix** is a powerful fusion of **ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes** and **invisietch/EtherealRainbow-v0.3-8B**, utilizing **SLERP (Spherical Linear Interpolation)** for optimal blending of embeddings. This merge enhances reasoning, contextual understanding, and creative language generation while retaining ethical alignment and responsiveness.

---

## 🔥 **Merged Models**
- **[ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes](#)** - A highly optimized instruction-tuned model, built for nuanced, long-form reasoning.
- **[invisietch/EtherealRainbow-v0.3-8B](https://huggingface.co/invisietch/EtherealRainbow-v0.3-8B)** - A dynamic conversational model with expanded alignment and expressiveness.

---

## ⚙️ **Merge Configuration**
The following YAML configuration defines how these models were fused using **SLERP**:

```yaml
# Merge configuration for ZeroXClem-Llama-3.1-8B-RainbowLight-EtherealMix using SLERP

name: ZeroXClem-Llama-3.1-8B-RainbowLight-EtherealMix
slices:
  - sources:
      - model: ZeroXClem/Llama-3.1-8B-SuperNova-EtherealHermes
        layer_range: [0, 32]
      - model: invisietch/EtherealRainbow-v0.3-8B
        layer_range: [0, 32]
merge_method: slerp
base_model: invisietch/EtherealRainbow-v0.3-8B
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

```

### **Why SLERP?**
- **Maintains Model Integrity**: Ensures a smooth transition between feature spaces of both models.
- **Preserves Semantic Meaning**: Avoids interpolation collapse, keeping token embeddings rich in structure.
- **Balanced Performance**: Retains the best qualities from both parent models.

---

## 🚀 **Capabilities**
### 🌟 **Enhanced Features**
- **Supercharged Instruction Following** – More intuitive and context-aware.
- **Advanced Conversational Flow** – Generates human-like responses with coherence.
- **Creative and Expressive Writing** – Ideal for storytelling, summarization, and content generation.
- **Expanded Knowledge Base** – Merging brings broader factual recall and conceptual understanding.
- **Flexible Alignment** – A balance of compliance and open-ended response generation.

---

## 📥 **Usage Instructions**
### **Transformers**
You can use the model via Hugging Face's `transformers` library:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Sample inference
prompt = "What are the implications of artificial intelligence in the future of education?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

### **Ollama**
For local execution with Ollama:
```sh
ollama run hf.co/ZeroXClem/Llama-3.1-8B-RainbowLight-EtherealMix
```


---

## 📌 **Important Notes**
- **License**: Governed by **Meta's Llama 3.1 Community License**.
- **Alignment Considerations**: Users are responsible for ethical and compliant use.
- **System Tokens**: Follows Llama 3.1 tokenization standards for inference stability.
- **Quantization**: **Use FP16 for optimal performance**, though **Q8** quantized versions may be available.

---

## 💜 **Special Thanks**
Deep gratitude to:
- **@invisietch** for EtherealRainbow-v0.3-8B.
- **Hugging Face & Open-Source AI Community** for their incredible contributions. 🚀💖

---

## 🔗 **Resources**
- **[Hugging Face Model Page](#)**
- **[Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)**
- **[MergeKit Repository](https://github.com/cg123/mergekit)**

---

**✨ Merged with precision. Optimized for excellence. Experience RainbowLight EtherealMix today! ✨**
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/ZeroXClem__Llama-3.1-8B-RainbowLight-EtherealMix-details)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |22.83|
|IFEval (0-Shot)    |49.73|
|BBH (3-Shot)       |31.07|
|MATH Lvl 5 (4-Shot)|12.16|
|GPQA (0-shot)      | 4.92|
|MuSR (0-shot)      | 9.87|
|MMLU-PRO (5-shot)  |29.23|