LLaMA-3.2-1B-Instruct Fine-Tuned for Student Mental Health Counseling
Model Overview
This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct, customized specifically for mental health and counseling tasks. It was trained to provide empathetic, safe, and context-aware responses for student well-being and mental health support.
It is optimized for usage in educational environments, AI-driven mental health chatbots, and therapeutic content generation.
Dataset
The model was trained on a curated and preprocessed mental health dataset:
arafatanam/Student-Mental-Health-Counseling-50K— translated/adapted from chillies/student-mental-health-counseling-vn (50,000 samples)
Training Configuration
- Framework: 🤗 Transformers +
UnslothLoRA adapter - Hardware: 2x NVIDIA T4 GPUs (Kaggle Notebooks)
- Precision: FP16 (with fallback for BFloat16)
- LoRA: Enabled (Low-Rank Adaptation fine-tuning)
- Tokenizer: AutoTokenizer (LLaMA-compatible)
Training Arguments
| Parameter | Value |
|---|---|
max_seq_length |
512 |
per_device_train_batch_size |
1 |
gradient_accumulation_steps |
8 |
num_train_epochs |
1 |
learning_rate |
2e-4 |
warmup_ratio |
0.01 |
optimizer |
adamw_8bit |
lr_scheduler_type |
cosine |
weight_decay |
0.01 |
max_grad_norm |
0.5 |
eval_steps |
200 |
save_steps |
1000 |
logging_steps |
100 |
Training Metrics
| Metric | Value |
|---|---|
| Train Loss | 1.0791 |
| Final Step Loss | 0.9961 |
| Training Time | 8,591.93 seconds |
| FLOPs (total) | 50.65 Trillion |
| Global Steps | 3,125 |
| Epochs | 1 |
| Samples/Second | 5.82 |
| Steps/Second | 0.36 |
| Gradient Norm | 0.5741 |
| Learning Rate (Final) | 3.49e-8 |
Use Cases
This model is optimized for:
- 🧑⚕️ AI Mental Health Chatbots
- 🧘 Self-help conversation agents
- 🎓 University/College student mental wellness systems
- 🧾 Therapeutic content generation
- 🗣️ Conversational AI for safe, guided emotional support
Limitations & Considerations
- This model is not a replacement for professional mental health care.
- Designed primarily for educational and support purposes in controlled environments.
- Although it has been fine-tuned for empathy and safety, human supervision is recommended for sensitive use cases.
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for arafatanam/KindMind-Llama-3.2-1B-Instruct
Base model
meta-llama/Llama-3.2-1B-Instruct