Facial expression recognition from images for various head poses -

Abstract

In the process of facial expression detection from image or video modalities, the variation of head poses with respect to the camera causes a challenging problem for any robust recognition. Several studies have been conducted on the effect of the pose on the recognition rate. The prevalent methodology to solve this problem consists of transforming the facial features back to a frontal pose before inferring the facial expression. Some work has further considered splitting the face into multiple parts then performing a simple maximum combination of the classifications. In this work, we propose a new approach for splitting and fusing the facial features in cases with head yaw rotations. The approach consists of splitting the face into left and right features. Then, two methods are proposed to classify the facial expression. In the first method, we detect facial Action Units (AUs) in the left and right parts then combine the results using a logical OR operation. In the second method, we propose an optimized fusion of the facial expressions. The outcome of the optimized method is a set of weights to combine the classifications from each side of the face at different yaw angles. The weights are determined dynamically based on the yaw angle of the head through a polynomial regression. Experiments were conducted on the two methods using a custom-made database and a set of benchmark 3D facial images. The results showed a 7.1percent improvement for our proposed split-and-fuse method over full facial features approach. Furthermore, the optimized fusion method showed superiority in comparison to max-based fusion.