metadata

license: llama2
base_model: codellama/CodeLlama-7b-Instruct-hf
tags:
  - fine-tuned
  - educational
  - qa
  - code
  - llama
  - peft
  - lora
language:
  - en
pipeline_tag: text-generation
library_name: peft

CodeLLaMa7B-FineTuned-byMoomen

This model is a fine-tuned version of codellama/CodeLlama-7b-Instruct-hf using LoRA (Low-Rank Adaptation) for educational Q&A tasks.

Model Details

Base Model: codellama/CodeLlama-7b-Instruct-hf
Fine-tuning Method: LoRA (Low-Rank Adaptation)
LoRA Rank: 32
LoRA Alpha: 64
Target Modules: ['gate_proj', 'lm_head', 'k_proj', 'q_proj', 'up_proj', 'down_proj', 'v_proj', 'o_proj']
Training Focus: Educational programming Q&A
Model Type: Causal Language Model

Usage

Quick Start

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

# Load model and tokenizer
model = AutoPeftModelForCausalLM.from_pretrained("Moomen123Msaadi/CodeLLaMa7B-FineTuned-byMoomen")
tokenizer = AutoTokenizer.from_pretrained("codellama/CodeLlama-7b-Instruct-hf")

# Generate response
prompt = "Explain recursion in programming"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=300, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Chat Format Usage

# For educational Q&A conversations
messages = [
    {"role": "system", "content": "You are a helpful educational assistant."},
    {"role": "user", "content": "What is the difference between lists and tuples in Python?"}
]

formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(formatted_prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=300)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

Memory-Efficient Loading

# For systems with limited VRAM
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16
)

model = AutoPeftModelForCausalLM.from_pretrained(
    "Moomen123Msaadi/CodeLLaMa7B-FineTuned-byMoomen",
    quantization_config=quantization_config,
    device_map="auto"
)

Training Details

This model was fine-tuned using:

Parameter-Efficient Fine-Tuning (PEFT) with LoRA
Educational conversation dataset focused on programming concepts
Optimized for Q&A format with system/user/assistant roles

Intended Use

This model is designed for:

📚 Educational programming Q&A
💡 Concept explanations in computer science
🔧 Code debugging assistance
🎓 Technical tutoring and learning support

Limitations

Based on codellama/CodeLlama-7b-Instruct-hf, inherits its limitations
Optimized for educational content, may not perform well on other tasks
Requires base model for inference (LoRA adapters only)
Performance depends on the quality of training data

Model Architecture

This is a LoRA adapter that needs to be loaded with the base model. The adapter files are:

adapter_config.json: LoRA configuration
adapter_model.safetensors: Trained LoRA weights

License

This model follows the same license as the base model: Llama 2 Custom License.

Citation

If you use this model, please cite:

@misc{CodeLLaMa7B_FineTuned_byMoomen,
  title={CodeLLaMa7B-FineTuned-byMoomen},
  author={Moomen123Msaadi},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/Moomen123Msaadi/CodeLLaMa7B-FineTuned-byMoomen}
}