😊 Facial Emotion Recognition

🧠 CNN-Powered Emotion Classifier built with Flask & PyTorch

✨ Overview

Facial Emotion Recognition is a sophisticated Flask web application that detects and classifies human emotions from facial expressions in real-time. Built with PyTorch CNNs and powered by advanced face detection algorithms, this application recognizes 7 distinct emotions with high accuracy across images, videos, GIFs, and live camera feeds.

The project seamlessly combines state-of-the-art deep learning with an intuitive, modern web interface featuring glassmorphism design patterns. Whether you're analyzing a static image or processing dynamic video content, our AI delivers instant, intelligent emotion predictions.

📦 Key Capabilities:

✔ Real-time emotion detection from live camera feeds
✔ Image analysis (PNG, JPG, JPEG, BMP)
✔ GIF frame processing & emotion tracking
✔ Video file support (MP4, AVI, MOV, MKV, FLV, WEBM)
✔ Confident predictions with top-3 emotion scores
✔ Automatic face detection & extraction
✔ Interactive web interface with instant feedback

All in milliseconds.

🎯 Demo Flow

Upload Image / GIF / Video / Stream from Live Camera
            ↓
Extract Face(s) using Haar Cascade Classifier
            ↓
Preprocess: Grayscale → Normalize → Resize (48×48)
            ↓
Feed into PyTorch CNN Model
            ↓
Predict Emotion + Confidence Score
            ↓
Display Result with Top-3 Emotions & Emoji

📸 Screenshots

💻 Default Interface

😮 Image Emotion Prediction (Surprise)

🎬 Video Emotion Analysis

Click the image above to watch the full demo video

🔥 Features

🎥 Multiple Input Modes

Live Camera Stream — Real-time emotion detection with continuous face tracking
Image Upload — Analyze single photographs for dominant emotions
Video Processing — Extract and analyze emotions across video frames
GIF Animation — Track emotional changes frame-by-frame
All modes return predicted emotion + confidence score + top-3 alternatives

🧠 PyTorch CNN Model

Architecture: 4 Convolutional blocks + 2 Dense layers
Training Data: FER2013 & custom datasets (thousands of facial images)
Layer Stack:
- Conv2D (32 filters) → BatchNorm → ReLU → MaxPooling → Dropout(0.25)
- Conv2D (64 filters) → BatchNorm → ReLU → MaxPooling → Dropout(0.25)
- Conv2D (128 filters) → BatchNorm → ReLU → MaxPooling → Dropout(0.3)
- Conv2D (256 filters) → BatchNorm → ReLU → MaxPooling → Dropout(0.3)
- Flatten → Dense(256) → BatchNorm → Dropout(0.5) → Dense(7) → Softmax
Output Classes: 7 emotions (Angry, Disgust, Fear, Happy, Neutral, Sad, Surprise)
Device Support: Automatic GPU/CPU detection (CUDA optimized)

🎯 Emotion Classification

Angry — Detected through furrowed brows & tensed jaw
Disgust — Nose wrinkle & lip curl patterns
Fear — Wide eyes & raised eyebrows signatures
Happy — Smile detection & crow's feet recognition
Neutral — Resting face with minimal expression
Sad — Downturned mouth & inner eyebrow raise
Surprise — Maximum eye/mouth opening signals

🖼️ Advanced Face Detection

Haar Cascade Classifier for robust face localization
Automatic face extraction & cropping
Multi-face support (processes first detected face)
Fallback to full image if no face detected
Works across diverse lighting conditions & angles

📊 Intelligent Preprocessing

Grayscale conversion for model optimization
Dynamic resizing to 48×48 pixel standard
Normalization with mean=0.5, std=0.5
Automatic frame sampling for long videos (≤200 frames)
Memory-efficient batch processing

💎 UI/UX Excellence

Glassmorphism Design — Modern frosted glass aesthetic
Gradient Animated Background — Smooth color transitions
Real-time Progress Visualization — Live confidence bars
Responsive Layout — Works seamlessly on mobile & desktop
Intuitive Controls — Drag-drop file upload, one-click camera
Emoji Feedback — Visual emotion representation
Dark Theme — Easy on the eyes, modern feel

⚡ Backend Performance

Flask lightweight web server
Real-time frame streaming capabilities
Efficient video/GIF decoding with OpenCV
File upload handling (≤16MB limit)
Asynchronous processing pipeline
CORS-compatible API endpoints

🧠 How It Works (Step-by-Step)

Step 1 — Extract & Detect Face

Raw Input (Image/Frame)
    ↓
Convert to Grayscale
    ↓
Apply Haar Cascade Classifier
    ↓
Locate Face Region (x, y, w, h)
    ↓
Extract Face ROI from Image

Step 2 — Preprocess Image

Extracted Face Image
    ↓
Convert to Grayscale (3-channel)
    ↓
Resize to 48×48 pixels
    ↓
Convert to PyTorch Tensor
    ↓
Normalize: (pixel - 0.5) / 0.5
    ↓
Add batch dimension (1, 3, 48, 48)

Step 3 — CNN Forward Pass

Input (1, 3, 48, 48)
    ↓
Conv2D(32) → BatchNorm → ReLU → MaxPool(2) → Dropout(0.25)
    ↓
Conv2D(64) → BatchNorm → ReLU → MaxPool(2) → Dropout(0.25)
    ↓
Conv2D(128) → BatchNorm → ReLU → MaxPool(2) → Dropout(0.3)
    ↓
Conv2D(256) → BatchNorm → ReLU → MaxPool(2) → Dropout(0.3)
    ↓
Flatten → Dense(256) → BatchNorm → Dropout(0.5)
    ↓
Dense(7) → Softmax
    ↓
Output Logits [7 classes]

Step 4 — Emotion Recognition

Model Output: [angry_score, disgust_score, fear_score, ...]
    ↓
Apply Softmax Normalization
    ↓
argmax(predictions)  →  Predicted Emotion
    ↓
max(predictions)  →  Confidence %
    ↓
Sort & Extract Top-3 Results
    ↓
Display with Emoji & Confidence Bar

🏗️ Tech Stack

Layer	Technology
Backend	Flask 2.3.2
Deep Learning	PyTorch 2.0.1, TorchVision 0.15.2
Face Detection	OpenCV 4.8.0 (Haar Cascade)
Image Processing	Pillow (PIL) 10.0.0, NumPy 1.24.3
Frontend	HTML5 + Bootstrap 5 + CSS3 + JavaScript
Styling	Custom CSS (Glassmorphism Pattern)
Fonts	Google Fonts (Poppins)
Icons	Font Awesome 6.0
Deployment	Werkzeug 2.3.6

📂 Project Structure

Facial-Emotion-Recognition/
│
├── app.py                              # Main Flask application
├── requirements.txt                    # Python dependencies
│
├── model/
│   ├── best_model.pth                  # Trained PyTorch model
│   └── classes.json                    # Emotion class mappings
│
├── notebooks/
│   └── Facial-Emotion-Recognition.ipynb # Jupyter training notebook
│
├── scripts/
│   ├── generate_classes.py             # Utility to create classes.json
│   ├── learn.py                        # Model training & evaluation script
│   └── predict.py                      # Standalone prediction utility
│
├── templates/
│   └── index.html                      # Web UI HTML template
│
├── static/
│   ├── style.css                       # Glassmorphism styling
│   └── script.js                       # Frontend interactivity
│
├── sampleScreenshots/
│   ├── Screenshot (18).png             # Default UI
│   └── Screenshot (19).png             # Emotion prediction result
│   └── videoThumbnail.png              # Demo video thumbnail
│
├── sampleVideo.mp4                     # Demo video with multiple emotions
│
└── README.md                           # This file

⚙️ Installation

1️⃣ Clone Repository

git clone https://github.com/SACHIN-S-2004/Facial-Emotion-Recognition.git
cd Facial-Emotion-Recognition

2️⃣ Create Virtual Environment (Recommended)

# Windows
python -m venv venv
venv\Scripts\activate

# macOS / Linux
python3 -m venv venv
source venv/bin/activate

3️⃣ Install Dependencies

pip install -r requirements.txt

Dependencies:

torch==2.0.1
torchvision==0.15.2
numpy==1.24.3
Pillow==10.0.0
Flask==2.3.2
opencv-python==4.8.0.74
Werkzeug==2.3.6

4️⃣ Run Application

python app.py

The server will start at: http://127.0.0.1:5000 🚀

5️⃣ Open in Browser

http://localhost:5000

🎓 How to Use

📷 Upload Image/GIF/Video

Click "Drag & drop your file" or click to browse
Select an image (PNG, JPG, JPEG, BMP), GIF, or video (MP4, AVI, MOV, etc.)
Click "Initialize Analysis" button
View predictions with confidence scores & top-3 emotions

🎥 Live Camera Detection

Click "Live Camera" button in navigation bar
Click "Start Camera" to begin real-time emotion detection
Express different emotions to see instant classification
Click "Stop Camera" to end session

🔍 Interpret Results

Top Emotion — Most confident prediction with percentage
Confidence Bar — Visual representation of certainty
Top-3 Alternatives — Secondary emotion possibilities
Emoji Indicator — Quick visual emotion representation

📊 Model Performance

Metric	Value
Input Size	48 × 48 pixels
Emotions	7 classes
Model Type	Convolutional Neural Network (CNN)
Framework	PyTorch
Optimization	Adam Optimizer
Loss Function	CrossEntropyLoss
Activation	ReLU + Softmax
GPU Support	CUDA-enabled
Inference Time	~50-100ms per frame

🎯 Supported Emotions

Emotion	Emoji	Characteristics
Angry	😠	Furrowed brows, tensed jaw
Disgust	🤢	Nose wrinkle, lip curl
Fear	😨	Wide eyes, raised eyebrows
Happy	😊	Smile, crow's feet
Neutral	😐	Minimal expression
Sad	😢	Downturned mouth
Surprise	😮	Maximum eye opening

🎨 UI Features

🌟 Glassmorphism Design

Modern aesthetic with semi-transparent, frosted glass effect cards and smooth animations.

🎬 Real-time Visualization

Live emotion detection in camera modal with split-screen layout showing video feed and results simultaneously.

📊 Confidence Visualization

Animated progress bars showing emotion confidence percentages with gradient styling.

🎯 Responsive Layout

Adapts seamlessly to different screen sizes (mobile, tablet, desktop) using Bootstrap grid system.

🎨 Color-coded Results

Visual distinction between emotions using emoji and color gradients for intuitive understanding.

🚀 Advanced Features

Video Frame Sampling

Automatically samples frames from long videos to optimize processing
Maximum 200 frames processed per video (configurable)
Maintains emotion tracking across temporal sequences

Multi-face Handling

Detects multiple faces in a single frame
Processes primary face for emotion classification
Fallback mechanisms for edge cases

Efficient Preprocessing

Grayscale conversion reduces computation
Standardized 48×48 resolution for consistency
Normalization ensures model stability

Real-time Streaming

Live camera feed with millisecond latency
Continuous face detection & emotion prediction
Smooth UI updates with minimal lag

🔧 Configuration

Modify Model Parameters

Edit app.py to adjust:

MAX_FRAMES = 200              # Max frames from video to process
MAX_CONTENT_LENGTH = 16MB     # Max file upload size
ALLOWED_EXTENSIONS = {...}   # Supported file types

📚 API Endpoints

POST `/predict`

Accepts image upload and returns emotion prediction

Request:

{
  "file": <image_file>
}

Response:

{
  "emotion": "happy",
  "confidence": 0.95,
  "top_3": [
    {"emotion": "happy", "score": 0.95},
    {"emotion": "surprise", "score": 0.03},
    {"emotion": "neutral", "score": 0.02}
  ]
}

🎓 Learning Outcomes

This project demonstrates:

✔ Deep Learning — CNN architecture design & training
✔ Computer Vision — Face detection & image preprocessing
✔ PyTorch — Model building, training, and inference
✔ Flask Development — Backend API creation & deployment
✔ Frontend Integration — Interactive web UI with real-time updates
✔ Video Processing — GIF & video frame extraction
✔ Model Optimization — GPU acceleration & efficient inference
✔ Full-Stack Development — End-to-end application architecture

🐛 Troubleshooting

No faces detected

Ensure good lighting conditions
Face should occupy ~30-40% of image frame
Try uploading a clearer portrait

Low confidence predictions

Model works best with frontal face views
Avoid extreme angles or side profiles
Ensure facial features are clearly visible

Slow inference

CPU processing is slower than GPU
Install CUDA for GPU acceleration
Reduce video frame count or resolution

Camera not working

Grant browser camera permissions
Check if another app is using the camera
Try a different browser

📝 Model Training

To retrain the model on custom dataset:

python scripts/learn.py

Refer to notebooks/Facial-Emotion-Recognition.ipynb for detailed training notebook.

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
data		data
model		model
notebooks		notebooks
sampleScreenshots		sampleScreenshots
scripts		scripts
static		static
templates		templates
.gitignore		.gitignore
README.md		README.md
app.py		app.py
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

😊 Facial Emotion Recognition

🧠 CNN-Powered Emotion Classifier built with Flask & PyTorch

✨ Overview

🎯 Demo Flow

📸 Screenshots

💻 Default Interface

😮 Image Emotion Prediction (Surprise)

🎬 Video Emotion Analysis

🔥 Features

🎥 Multiple Input Modes

🧠 PyTorch CNN Model

🎯 Emotion Classification

🖼️ Advanced Face Detection

📊 Intelligent Preprocessing

💎 UI/UX Excellence

⚡ Backend Performance

🧠 How It Works (Step-by-Step)

Step 1 — Extract & Detect Face

Step 2 — Preprocess Image

Step 3 — CNN Forward Pass

Step 4 — Emotion Recognition

🏗️ Tech Stack

📂 Project Structure

⚙️ Installation

1️⃣ Clone Repository

2️⃣ Create Virtual Environment (Recommended)

3️⃣ Install Dependencies

4️⃣ Run Application

5️⃣ Open in Browser

🎓 How to Use

📷 Upload Image/GIF/Video

🎥 Live Camera Detection

🔍 Interpret Results

📊 Model Performance

🎯 Supported Emotions

🎨 UI Features

🌟 Glassmorphism Design

🎬 Real-time Visualization

📊 Confidence Visualization

🎯 Responsive Layout

🎨 Color-coded Results

🚀 Advanced Features

Video Frame Sampling

Multi-face Handling

Efficient Preprocessing

Real-time Streaming

🔧 Configuration

Modify Model Parameters

📚 API Endpoints

POST /predict

🎓 Learning Outcomes

🐛 Troubleshooting

No faces detected

Low confidence predictions

Slow inference

Camera not working

📝 Model Training

🤝 Contributing

⭐ If you like this project

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

POST `/predict`

Packages