Mood Detection: Implementing a facial expression recognition system

 Neeraj Agrawal, Rob Cosgriff and Ritvik Mudur

Introduction

The purpose of this research is to develop a mood detection system which can achieve high recognition rates of facial expression across multiple databases.

Approach

An approach using eigenface masks was developed and implemented to classify facial expressions across publicly available databases. These learned masks along with feature vectors of the image were used to train the SVM (Support Vector Machines).

Method

• Pre-processing image data by normalizing and applying a simple mask.
• Extracting certain (facial) features using PCA (Principal Component Analysis) and Gabor filters.
• Using SVMs for classification and recognition of expressions.

Feature extraction

a.     Normalized pixel intensities (Every image in training set normalized by subtracting the mean of all training set images. The masked region is then converted to a column vector which forms the feature vector).

b.     Gabor filter representations (Gabor filters are often used in image processing and are based on physiological studies of the human visual cortex. The use of Gabor filtered facial images has been shown to result in improved accuracy for facial expression recognition).

Eigenface masks (eigenfaces are used to generate a mask that eliminates pixels that vary little across training samples in different labels. In this system, they modify the approach to generate a separate mask for each expression class).

Strength

When apply Eigenface mask the results were similar for both the normalized pixel value and the gabor filtered feature vectors.

Advantages

• This approach works reasonably well in capturing expression-relevant facial information across all databases.

Conclusion

The results were similar for both the normalized pixel value and the gabor filtered feature vectors, with neither representation being clearly superior to the other. The performance across databases is indicative of the method’s robustness to variations in facial structure and skin tone when recognizing the expression.

However, both representations of the feature vectors end up being very high dimensional. This hurts the run time of the algorithm hampering the ability to use it in real time, as well as leading to possible over fitting of the data during training. Increasing the number of available training images would help to compensate for over fitting.

Download Research Paper

http://cs229.stanford.edu/proj2009/AgrawalCosgriffMudur.pdf