mustafij48/qwen2.5-0.5B_on_databricks_dolly_4k_datasize_5_epoch

Model Description

This model is obtained by training QWEN2.5-0.5B model with 4000 data from databricks-dolly-15k dataset for 5 epochs. A small number of LoRA adapters were only trained with PEFT and LoRA to get the model.

How to Get Started with the Model

Use the code below to get started with the model.

import the model

from peft import PeftModel

# Load model and tokenizer
base_model_name = "Qwen/Qwen2.5-0.5B"
model_name = "mustafij48/qwen2.5-0.5B_on_databricks_dolly_4k_datasize_5_epoch"
base_model = AutoModelForCausalLM.from_pretrained(base_model_name, force_download = True)
model = PeftModel.from_pretrained(base_model, model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name, force_download = True)
tokenizer.pad_token = tokenizer.eos_token

set device

import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"using device {device}")
model = model.to(device)

Perform inference

input_text = f"Write me a list of five fantacy novels",
inputs = tokenizer(input_text, return_tensors="pt").to(device)
outputs = model.generate(**inputs, max_length=128)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))