mishbahulal/distilbert-base-uncased

Emotion Classification Model 🤖💬

A BERT-based emotion classification model fine-tuned on the GoEmotions dataset to detect emotions in English text.

The model predicts emotional categories from input sentences and can be used for sentiment analysis, emotion detection, and conversational AI systems. version of the BERT base model. It was introduced in this paper. The code for the distillation process can be found here. This model is uncased: it does not make a difference between english and English.

DistilBERT base model (uncased) This model is a distilled version of the BERT base model. It was introduced in this paper. The code for the distillation process can be found here. This model is uncased: it does not make a difference between english and English.

Model Details

Property	Value
Base Model	google-bert/bert-base-uncased
Task	Text Classification
Dataset	GoEmotions
Framework	Hugging Face Transformers
Language	English
License	MIT

---

Dataset

This model is trained on the GoEmotions dataset, a large dataset of human-annotated emotions extracted from Reddit comments.

Dataset Characteristics

• 58,000+ Reddit comments
• 27 emotion labels
• Multi-label emotion classification

Dataset Link: https://huggingface.co/datasets/google-research-datasets/go_emotions

---

Installation

pip install transformers torch

---

Usage 🚀

Using the Hugging Face Pipeline

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="mishbahulal/distilbert-base-uncased"
)

classifier("I am extremely happy today!")

Example Output

[
  {
    "label": "joy",
    "score": 0.97
  }
]

---

Manual Inference

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "mishbahulal/distilbert-base-uncased"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "I feel amazing today!"

inputs = tokenizer(text, return_tensors="pt")

outputs = model(**inputs)

prediction = torch.argmax(outputs.logits, dim=1)

print(prediction)

---

Training Hyperparameters

The following hyperparameters were used during training:

Hyperparameter	Value
Learning Rate	2e-05
Train Batch Size	64
Eval Batch Size	64
Epochs	2
Random Seed	42
Optimizer	AdamW Torch Fused
Betas	(0.9, 0.999)
Epsilon	1e-08
Learning Rate Scheduler	Linear

Optimizer configuration:

OptimizerNames.ADAMW_TORCH_FUSED
betas=(0.9,0.999)
epsilon=1e-08
optimizer_args=None

---

Training Results

(Add evaluation metrics here if available)

Metric	Value
Accuracy	(add result here)

---

Framework Versions

The model was trained using the following library versions:

Library	Version
Transformers	5.0.0
PyTorch	2.10.0+cu128
Datasets	4.0.0
Tokenizers	0.22.2

---

Intended Uses

This model can be used for:

• Emotion detection
• Sentiment analysis
• Social media monitoring
• Chatbots with emotional awareness
• Customer feedback analysis

---

Limitations ⚠️

• Trained only on English language data
• May inherit bias from Reddit-based datasets
• Performance may vary on domain-specific text (medical, legal, etc.)

---

Ethical Considerations

Emotion classification models should be used responsibly. They should not be used for psychological diagnosis or sensitive decision-making systems.

---

Citation

If you use this model, please cite the GoEmotions paper:

@inproceedings{demszky2020goemotions,
title={GoEmotions: A Dataset of Fine-Grained Emotions},
author={Demszky et al.},
year={2020}
}

---

Model Maintainer

Created by Mishbahul Al Islam

Contributions and feedback are welcome.