YimingZeng/FineEdit_Model

The following example illustrates how to load the FineEdit-XL adapter for inference. For FineEdit-Pro, we use Qwen2.5-3B-Instruct as the base model.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

## Base Model
base_model = "meta-llama/Llama-3.2-3B"

## Lora
adapter_model = "YimingZeng/FineEdit_Model"

subfolder = "FineEdit-XL"

## Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(base_model)
base = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

## Load LoRA adapter
model = PeftModel.from_pretrained(base, adapter_model, subfolder=subfolder)

## Test
prompt = """Edit Request: Please change 'Captain American' to 'Iron Man'.
Original Content: 'Captain American' is a superhero.
Edited Content:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=100)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))