The benefits of synthetic data for action categorization.

Abstract

In this thesis, we will show the importance of video analysis using deep network. We are going to introduce some deep learning methods to detect and recognize faces in a video stream as well as emotion recognition. The three preceding networks are based on image analysis systems. Another way of analyzing videos is to do action recognition system, where the order of the frames becomes important. We propose a new 3DCNN+LSTM system for action recognition. However, the proposed system did not outperform the state of the art systems on UCF-101 dataset. In fact, it scored around 80 percent on UCF-101 while the state of the art system scores above 90 percent. In addition, we studied the value of using synthetically produced videos as training data for neural networks used for action categorization. Motivated by the fact that texture and background of a video play little to no significant roles in optical flow, we generated simplified texture-less and background-less videos and utilized the synthetic data to train a Temporal Segment Network (TSN). The results demonstrated that augmenting TSN with simplified synthetic data improved the original network accuracy (68.5percent), achieving 71.8percent on HMDB-51 when adding 4,000 videos and 72.4 percent when adding 8,000 videos. Also, training using simplified synthetic videos alone on 25 classes of UCF-101 achieved 30.71 percent when trained on 2500 videos and 52.7 percent when trained on 5000 videos. Finally, results showed that when reducing the number of real videos of UCF-25 to 10 percent and combining them with synthetic videos, the accuracy drops to only 85.41 percent from 96.60 percent, compared to a drop to 77.4 percent when no synthetic data is added.