Model Name - L3.3-70B-Animus-V1

Model Description

This is a fine-tuned version of SicariusSicariiStuff/Negative_LLAMA_70B specialized for roleplaying and instruction-following within the Wings of Fire universe.

The model was trained on a 3-million-token dataset extracted from the Wings of Fire book series to build a strong foundation of domain knowledge. It was fine-tuned for 2 epochs on a curated dataset of conversational examples.

The goal of this model is to provide a high-quality, immersive, and lore-accurate conversational experience for fans of the series. It can adopt character personas, answer questions about the world, and engage in creative storytelling.

Training Details

Training Hardware

The model was trained on a single NVIDIA H200 GPU.

Training Procedure

A QLoRA (Quantized Low-Rank Adaptation) approach was used for efficient fine-tuning.

Training Data

The training process involved two main stages:

1. Domain training: The base model was adapted to the Wings of Fire universe using a custom dataset of 3 million tokens compiled directly from the book series. This step saturated the model with the specific lore, characters, and writing style of the source material.

2. Instruction & Chat Fine-tuning: The model was fine-tuned on a mixed dataset of 2,200 examples:

   • 1,400 Roleplay Conversations: Multi-turn conversational examples designed to teach the model how to adopt and maintain character personas from the series.
   • 800 Assistant Examples: Instruction-response pairs focused on answering lore questions and following commands within the context of the Wings of Fire world.

Intended Use & Limitations

• Intended Use: This model is intended for creative and roleplaying purposes within the Wings of Fire universe. It is designed for fans of the series and is not a general-purpose chatbot.

• Limitations & Quirks:

  • Performance on tasks outside of its training domain (general knowledge, coding, etc.) is not guaranteed and will likely be poor.
  • The model may "hallucinate" or generate plausible but non-canonical information.
  • Content: The base model (Negative_LLAMA_70B) has a slight NSFW inclination. While this finetune was not trained for NSFW content, some of that behavior may bleed through. It is not the intended purpose of this model.
  • Formatting: The model has a strong tendency to use asterisks (*...*) for actions and emphasis. This is a learned behavior from its training data.
  • Safety: This model has not undergone additional safety alignment beyond what was included in its base. Standard responsible AI practices should be followed.

Acknowledgements

• Credit to Meta for the powerful Llama 3 architecture.
• Credit to SicariusSicariiStuff for the Negative_LLAMA_70B base model.