Model Card for LoLlama-3.2-1B-lora-3ep-v3-instruct

This model is a specialised fine-tuned LLM on League of Legend

This model is an instruct fine-tuned of the model avinot/LoLlama3.2-1B-lora-3ep-v3 that is a CLM LoRA fine-tuned version of the model meta-llama/Llama-3.2-1B

This model has been trained using TRL.

This model is currently not open-sourced as some benchmark needs to be performed before the full-release

Quick start

from transformers import pipeline
import torch

# Analysing which device is available
if torch.cuda.is_available():
  device = "cuda"
else:
  device = "cpu"

question = "What is the purpose of Thresh support ?"
generator = pipeline("text-generation", model="avinot/LoLlama-3.2-1B-lora-3ep-v3-instruct", device=device, token="[your_hf_read_token_here]")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with SFT.

Training result

Evolution of the train and eval loss

Evolution of the mean token accuracy on the validation and train splits

Framework versions

• TRL: 0.15.2
• Transformers: 4.49.0
• Pytorch: 2.6.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

Citations

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}