README.md

metadata

tags:
  - transformers
  - causal-lm
  - text-generation
  - instruct
  - chat
  - fine-tuned
  - merged-lora
  - llama-3
  - hermes
  - discord-dataset
  - conversational-ai
  - chatml
  - pytorch
  - open-weights
  - 3b-parameters
  - abliterated
base_model:
  - NousResearch/Hermes-3-Llama-3.2-3B-abliterated
model-index:
  - name: Discord-Micae-Hermes-3-3B
    results: []
datasets:
  - mookiezi/Discord-OpenMicae
library_name: transformers
license: mit

Try this model on Google Colab for free:

Discord-Micae-Hermes-3-3B-abliterated

Model Description

This is an uncensored version of mookiezi/Discord-Micae-Hermes-3-3B created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

Generation

from transformers import pipeline

gen = pipeline("text-generation", model="mookiezi/Discord-Micae-Hermes-3-3B-abliterated")
print(gen(
    "<im_start>user\nwhat do you do?<|im_end|>",
    max_new_tokens=100
))

Discord-Micae-Hermes-3-3B Model Description

Discord-Micae-Hermes-3-3B is a new finetune on NousResearch/Hermes-3-Llama-3.2-3B.

The model was trained on 17 million tokens of 250 thousand Discord STX (single turn exchanges) for 6 epochs and 5.5 million tokens of 100 thousand multi-turn chains for 6 epochs at learn rate 2e-5, finishing with both datesets combined for 1 epoch at 1e-5. We used a cosine warmup with 220 warmup steps for each phase. The LoRA adapter was trained with alpha = 32 and r = 8.

Dataset

This model was fine-tuned on the mookiezi/Discord-OpenMicae dataset.

Prompting

Micae uses the same ChatML as the prompt format as Hermes 3 and handles context.

<|im_start|>user
what do you think about ai?<|im_end|>
<|im_start|>assistant
It's cool, but I don't know what it is.<|im_end|>
<|im_start|>user
where do you think it's heading?<|im_end|>
<|im_start|>assistant
I can only speculate and make guesses.<|im_end|>
<|im_start|>user
what do you do use it for?<|im_end|>
<|im_start|>assistant
For fun and to learn new things.<|im_end|>
<|im_start|>user
like what?<|im_end|>
<|im_start|>assistant
Like playing games or building robots.<|im_end|>
<|im_start|>user
do you like pokemon?<|im_end|>
<|im_start|>assistant
Yes, but not the actual game.<|im_end|>
<|im_start|>user
what do you mean?<|im_end|>
<|im_start|>assistant
The idea behind the game, that a person could create an entire world from nothing.<|im_end|>