Model Card for NilayR/llama2-7b-backward-instruction

Model Details

Model Description

This model is a Llama-2-7b-chat-hf base model that has been fine-tuned to predict instructions from outputs. It serves as the $p(x|y)$ model in the "Self-Alignment with Instruction Backtranslation" framework, where 'x' represents the instruction and 'y' represents the output. This backward capability is crucial for generating synthetic instruction-output pairs for further instruction tuning.

• Developed by: Nilay Raut
• Model type: Causal Language Model (Fine-tuned)
• Language(s) (NLP): English
• License: Llama 2 Community License
• Finetuned from model: NousResearch/Llama-2-7b-chat-hf

Model Sources

• Paper: https://arxiv.org/pdf/2308.06259.pdf

Uses

Direct Use

This model is primarily intended for research purposes within the context of instruction backtranslation and self-alignment techniques for large language models. Its direct use involves generating instructions when given an output.

Out-of-Scope Use

This model is not intended for general-purpose conversational AI or direct user interaction. Its performance in generating human-like responses to direct prompts is not its primary function and may be limited compared to models fine-tuned for such tasks. It should not be used in production environments without further fine-tuning and safety evaluations.

Bias, Risks, and Limitations

This model inherits the biases and limitations of its base model, Llama-2-7b-chat-hf. Additionally, as a backward model, its generated instructions may sometimes be nonsensical or not perfectly aligned with human intent, especially for complex or ambiguous outputs. The quality of generated instructions is dependent on the diversity and quality of the initial training data.

Recommendations

Users should be aware that the generated instructions might require manual curation or additional filtering for quality control. It is recommended to evaluate the generated instructions for relevance, coherence, and safety before using them for downstream tasks.

How to Get Started with the Model

Use the code below to load the model and tokenizer from Hugging Face.

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load the base model and tokenizer
base_model_id = "NousResearch/Llama-2-7b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

# Load the fine-tuned LoRA adapter
model_id = "NilayR/llama2-7b-backward-instruction" # This is your uploaded model
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    load_in_4bit=True, # Or adjust based on your GPU
    device_map="auto"
)
model = PeftModel.from_pretrained(model, model_id)
model = model.merge_and_unload() # Merge LoRA weights for inference if desired
model.eval()

# Example usage to generate an instruction from an output
output_text = "Getting started in astrophotography can seem daunting, but with some patience and practice, you can become a master of the craft. To begin, you'll need a good camera and lens, a tripod, and a dark sky location free of light pollution. You will also need to learn about the basics of astro..."
input_text = f"### Output:\n{output_text}\n\n### Instruction:"

inputs = tokenizer(input_text, return_tensors="pt", truncation=True, max_length=256)
if torch.cuda.is_available():
    inputs = {k: v.to('cuda') for k, v in inputs.items()}

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=30,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        pad_token_id=tokenizer.pad_token_id,
        eos_token_id=tokenizer.eos_token_id,
        use_cache=True
    )

generated_instruction = tokenizer.decode(
    generated_ids[0][inputs['input_ids'].shape[1]:],
    skip_special_tokens=True
).strip()

print(f"Generated Instruction: {generated_instruction}")

Training Details

Training Data

The model was fine-tuned on a subset of the OpenAssistant Guanaco training dataset. This dataset consists of human-written instruction-output pairs. For this backward model, the data was re-formatted as ### Output:\n{output}\n\n### Instruction:\n{instruction}. A total of 3000 examples were used for training the backward model.

Training Procedure

Training Hyperparameters

• Training regime: Mixed precision (fp16) was used on GPU with 4-bit quantization.
• LoRA r: 8
• LoRA alpha: 16
• LoRA dropout: 0.05
• max_length: 256
• batch_size: 1
• gradient_accumulation_steps: 4
• max_steps: 150
• learning_rate: 3e-5
• warmup_steps: 10
• optim: paged_adamw_8bit (for GPU)

Model Card Contact

Nilay Raut