Notes

This prototype was taken from Kaggle, and it focuses on the nature or mood of person using video and shows real time facial expression. This can be helpful to predict whether the person is mentally healthy, sad, angry, or not. Analyzing behavioral and facial pattern of person, companies can use it to know whether their employees are happy to work or not. All the process that was done on that prototype are explained below in sequential order. This system can detect maximum 7 human emotion patters and the model was created using python and then it was converted into web platform by using flask framework.

1. Dataset Overview
Dataset was taken from internet that were available publicly( opensource) for tranning and validating the detection model. The dataset contains total images set of 35,889 samples with seven different facial expressions with tranning dataset 80%, validation dataset 10% and testing dataset 10%. When the dataset was imported using pandas, it was reshaped, and its header values were shown. The tranning, validation and testing data sets were then divided. Target levels were created, and all facial emotions were then placed on emotion map array. The occurrence of each emotion on the dataset were extracted and plotted further on graph showing the class distribution, total number of data and emotions to get clear overview of dataset.

Figure 1 Dataset Overview

2. Import Libraries
All the libraries were imported like numpy, pandas etc. Numpy was used for mathematical operations that contains values stored in the form of array. Pandas is used for data analysis and importing datasets in the form of either CSV or JSON file.

Figure 2 Import Libraries.
3. Detection Model
The image was then passed through different preprocessing methods where the image was resized, converted into array, and emotions were converted into pixels for depth analysis and to remove extra distortion or noise present in image.

The data was then split into 3 parts as explained in dataset overview section and the strings were converted to list of intigers. Images were reshaped into a dimension of 48x48, and grayscale was normalized to 255.0 followed by encoding label.



Each emotion that was kept on emotion_labels array was set bar labels and number. Then graph was plotted for each train, validation, and test dataset. Then the detection CNN architecture was created. Workflow of model is displayed as below:

After that model was analyzed under tranning phase. The accuracy, epoch, model accuracy and model loss were then plotted as graph for easy visualization.


The test performance was then evaluated. And overall accuracy of system was 66%. This is because the tranning time of model was less. The accuracy is directly proportional to tranning time.
4. Flask Implimentation
The same process was then repeated with slight changes like class, object was created, and images were converted as form data so that it will be easy to send and receive the input data and detected output i.e., expression name. The model was loaded from JSON file as shown in below image.


All the routes were managed on main.py. Here the libraries were imported, and the location was set for each route. When the flask server (local server ip address: 127.0.0.1)opens, the system will check if the index page was opened on browser or not. If not, then they system will stop after some time. When the index page was opened, camera was detected. Then the frame was created around human face and motion was tracked. If the human face moves, then the frame also moves with that face.