HenriAI - QLoRA-tuned GPT-J

This model is a QLoRA-tuned version of GPT-J-6B, trained on a diverse dataset including commonsense reasoning, academic essays, and natural conversations. It uses 4-bit quantization for efficient deployment while maintaining performance.

Model Details

• Base Model: EleutherAI/gpt-j-6b
• Training Technique: QLoRA (Quantized Low-Rank Adaptation)
• Language: English
• License: Same as base model (Apache 2.0)

Training Configuration

LoRA Parameters

• Rank (r): 72
• Alpha: 144
• Dropout: 0.1
• Target Modules: q_proj, v_proj, k_proj, out_proj, fc_in, fc_out
• Bias: none

Quantization Settings

• 4-bit quantization (NF4)
• FP16 compute dtype
• Double quantization enabled

Training Process

• Batch Size: 28
• Learning Rate: 3e-4
• Maximum Sequence Length: 512
• Number of Epochs: 5
• Gradient Accumulation Steps: 1
• Optimizer: AdamW with Cosine Annealing LR

Training Data

The model was trained on multiple datasets including:

• Commonsense reasoning data
• Academic essays
• Instruction sets
• Natural language conversations
• Introduction and conclusion examples

Usage

To use this model, format your inputs as:

prompt = f"Question: {your_question}\nAnswer:"

Recommended Generation Settings

generation_config = {
    "max_length": 512,
    "temperature": 0.7,
    "do_sample": True,
    "use_cache": True,
    "num_return_sequences": 1
}

Limitations

• Inherits base model biases from GPT-J-6B
• Optimized for Q&A format conversations
• May require 4-bit quantization support for inference

Technical Requirements

transformers>=4.31.0
peft
bitsandbytes>=0.39.0
torch>=2.0.0
accelerate

Citation & Contact

If you use this model or want to learn more about it, please contact me at [email protected]