LLaMA-3.2-3B-Instruct Fine-Tuned for Student Mental Health Counseling

Model Overview

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct, customized to support mental health and counseling conversations. It is adapted to respond compassionately and contextually to student mental health needs, suitable for AI chatbots, support tools, and educational assistance platforms.

Dataset

Fine-tuned using two merged and preprocessed datasets designed for mental health support:

arafatanam/Student-Mental-Health-Counseling-50K — Student-focused counseling data (50,000 samples)

Training Configuration

Framework: 🤗 Transformers + Unsloth + LoRA
Hardware: Dual NVIDIA T4 GPUs (Kaggle Notebooks)
Fine-tuning Technique: Parameter-efficient fine-tuning with LoRA
Tokenizer: LLaMA-compatible tokenizer
Precision: FP16 (with fallback for BF16 if unsupported)

Training Arguments

Argument	Value
`max_seq_length`	512
`per_device_train_batch_size`	1
`gradient_accumulation_steps`	8
`num_train_epochs`	1
`learning_rate`	2e-4
`warmup_ratio`	0.01
`optimizer`	adamw_8bit
`lr_scheduler_type`	cosine
`weight_decay`	0.01
`max_grad_norm`	0.5
`eval_steps`	200
`save_steps`	1000
`logging_steps`	100

Training Metrics

Metric	Value
Train Loss (Avg)	5.1548
Final Step Loss	3.7192
Training Time	24,210.73 seconds
FLOPs (Total)	146.28 Trillion
Global Steps	3,125
Epochs	1
Samples/Second	2.065
Steps/Second	0.129
Gradient Norm	374.22
Final Learning Rate	8.67e-8

Use Cases

This model is suitable for:

🤖 AI-based mental health chatbot platforms
🧘 Student well-being assistants
📚 Mental health education tools
🗣️ Conversational agents for emotional support
🧾 Therapeutic and wellness content generation

Limitations & Considerations

This model does not replace professional mental health services.
Best suited for non-clinical support tools in schools, universities, or awareness platforms.
Use in real-world applications should include human moderation for critical or sensitive cases.

arafatanam
/

AcademiMind-Llama-3.2-3B-Instruct