BAFCS/Emotion-Recognition-in-Education-through-Deep-Learning

This set contains two models trained in the context of a thesis on emotion recognition in education, using deep learning techniques.

• FacialModel-8177.h5: This model was developed as an ensemble combining convolutional neural networks, the VGG19 model, and Keras Tuner techniques. It was trained using two widely recognized datasets in the field of emotion recognition: FER2013 and CK+48. To address class imbalance, two complementary strategies were applied: (1) the use of weighted classes to adjust the importance of each class based on its representation, and (2) the generation of synthetic data through data augmentation techniques, which improved the distribution of emotions and the model’s generalization.

• TextModel-0.7836.h5: Model based on the RoBERTa architecture, extended with an N-Gram approach and fine-tuned using three Conv1D layers with 128 filters. The model was primarily trained on the ISEAR dataset and supplemented with two additional emotions —Surprise and Neutral— from the MELD dataset, selected for their relevance in educational contexts and supported by relevant scientific literature. Data augmentation techniques were applied to enhance dataset diversity, along with a thorough cleaning process to ensure the semantic quality of the texts.

Both models are in .h5 format and were developed using TensorFlow and Keras. They have been fine-tuned to capture emotional expressions relevant to educational environments.

Note on Model Loading:

• The FacialModel can be loaded using standard Keras methods:

  model = tf.keras.models.load_model("FacialModel-8177.h5")
  

• The TextModel uses custom objects (e.g., TFRobertaModel). You must specify them when loading the model:

  from transformers import TFRobertaModel
  from tensorflow.keras.models import load_model
  
  def custom_objects_fn_roberta():
        return {'TFRobertaModel': TFRobertaModel}
  
  model = load_model("TextModel-0.7836.h5", custom_objects=custom_objects_fn_roberta())
  

Source Code Repository

The complete source code for training, preprocessing, and evaluating the models can be found in the following repository: GitHub: BAFCS/Emotion-Recognition-in-Education-through-Deep-Learning

Contact: [email protected], [email protected]