OSCARcr/mistral-medquad-lora-r4

Mistral 7B – LoRA r=4 (MedQuAD)

Fine-tuned with LoRA (r=4) using dataset lavita/MedQuAD
Base model: mistralai/Mistral-7B-Instruct-v0.3
Quantization: 4-bit NF4
GPU: A100 80GB

📊 Final Results

• Validation Loss: 0.8431
• Perplexity: 2.324

🧾 Training Metrics

Epoch	Step	Training Loss	Validation Loss
0.33	200	0.855	0.852
0.66	400	0.821	0.826
1.0	600	0.807	0.812

💻 Example usage

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("REPO_ID", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")

text = "[INST] What is anemia? [/INST]"
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=80)
print(tokenizer.decode(out[0], skip_special_tokens=True)