Abstract:

This project aims to develop a real-time face emotion recognition system using a custom-trained Convolutional Neural Network (CNN). It leverages OpenCV for image capture and face detection, and TensorFlow/Keras for model training and prediction. The system allows capturing and labelling emotion-based face images, training a CNN model, and recognizing emotions from live webcam feeds. The approach ensures adaptability by training on custom datasets and supports emotions like happy, sad, angry, neutral, surprise, fear, and disgust.

Introduction:

Facial emotion recognition is a growing field in human-computer interaction, with applications in healthcare, education, customer service, and security. The goal is to interpret human emotions from facial expressions in real-time. By combining image processing techniques with deep learning, this project builds a custom pipeline to collect data, train a model, and deploy it for live emotion detection.

Problem Statement:

Existing systems for facial emotion recognition often rely on pre-trained models or fixed datasets, which may not generalize well to real-world scenarios. They also lack flexibility in customizing the emotion set and adapting to user-specific faces. A more interactive system that allows dataset creation and dynamic model training is needed.

Existing System and Disadvantages:

Existing System:

• Uses pre-trained emotion recognition models (e.g., FER2013, AffectNet).
• Limited to fixed emotion classes and datasets.
• General-purpose models may misclassify due to domain differences.

Disadvantages:

• Low accuracy in personalized settings.
• Inflexibility in adding custom emotion classes.
• Inability to retrain or fine-tune on user-specific data.

Proposed System and Advantages:

Proposed System:

• Enables users to capture and label images for various emotions.
• Trains a CNN model on the user-generated dataset.
• Recognizes emotions from live webcam video feed using OpenCV.

Advantages:

• User-customized training for better accuracy.
• Real-time inference with visual feedback.
• Flexible, extendable architecture.
• Full control over dataset and model behaviour.

Modules:

1. Data Collection Module:
   • Captures face images for selected emotion.
   • Automatically labels and stores them in folders.
2. Model Training Module:
   • Trains a CNN on the collected grayscale 48×48 images.
   • Validates and saves the model.
3. Realtime Emotion Detection Module:
   • Loads the trained model.
   • Detects faces and predicts emotion in real-time video.

Algorithms / Model:

• Face Detection: Haar Cascade Classifier (OpenCV)
• Emotion Classification Model:
  • Convolutional Neural Network (CNN) architecture:
    • Conv2D → MaxPooling → Conv2D → MaxPooling → Flatten → Dense → Dropout → Output
  • Activation: ReLU (hidden layers), Softmax (output)
  • Loss: Categorical Crossentropy
  • Optimizer: Adam

Software Requirements:

• Python 3.x
• OpenCV
• TensorFlow/Keras
• NumPy, Matplotlib, Seaborn
• OS: Windows/Linux

Hardware Requirements:

• Minimum 4 GB RAM
• Integrated/External Webcam
• GPU (optional, for faster training)

Conclusion:

This project successfully demonstrates a real-time facial emotion recognition system that is fully customizable. By capturing personal datasets and training from scratch, it offers a high degree of flexibility and adaptability. It ensures accurate emotion detection suitable for applications in AI-powered interaction systems.

Future Enhancement:

• Add support for multiple faces in a single frame.
• Integrate deep learning-based face detection (e.g., MTCNN).
• Improve performance using transfer learning (e.g., MobileNet, ResNet).
• Deploy as a web application or mobile app.
• Add logging or analytics dashboard for emotion trends.