DOMAIN ADAPTATION NEURAL NETWORKS FOR TIME SERIES CLASSIFICATION

Abstract

In this thesis, we solve two problems related to time-series prediction and domain adaptation (DA). For the first problem, we focus on investigating robust models for time-series prediction with application to epilepsy. Epilepsy is a chronic medical condition that involves abnormal brain activity causing patients to lose control of awareness or motor activity. As a result, detection of pre-ictal states, before the onset of a seizure, can be life-saving. The problem is challenging since it is difficult to discern between EEG signals in pre-ictal states versus signals in normal inter-ictal states. There are three key challenges that have not been previously addressed:(1) the inconsistent performance of prediction models across patients, (2) the lack of perfect prediction to protect patients from any episode, and (3) the limited amount of pre-ictal labeled data for advancing machine learning (ML) methods. The first part of the thesis addresses these limitations through a novel approach that uses adversarial examples with optimized tuning of a combined Convolution Neural Network (CNN) with Gated Recurrent Unit (GRU). Experiments showed that our new proposed solution achieved state of the art. Compared to previous state of the art, the results showed an improvement of 3x in model robustness as measured in reduced variations with area under the curve (AUC) and superior AUC accuracy with an average increase of 6.7%. In the second part of the thesis, we build on the success of the hybrid CNN-GRU model and investigate the problem of adapting models that have been trained for one source domain to a new target domain. When developing machine learning (ML) algorithms, it is commonly assumed that the training and testing data follow the same probability distribution. However, in real-world scenarios, non-stationary environments are more typical in applications such as Internet of Things (IoT) and wearables where the contexts frequently change over time. The problem can be formulated as domain adaptation (DA), where the settings of the training labeled data represent the source domain, and the unlabeled test data represent the target domain. The goal of DA is to develop a model that can predict the labels for data in the target domain. The idea is to have one model for source domain and another target domain model that can learn from the source. There has been extensive research on DA for learning domain invariant features. However, those methods remained limited in several aspects when considering advances for time series. Learning between source and target has relied on either using hard parameter sharing limiting the source and target models to be identical or using separate models but making an assumption of a linear relation between source and target parameters. The second open challenge is the model's limited ability to generalize to unseen data for both source and target. The third challenge is ensuring the proper choice of loss function for time-series DA. To address these challenges, we propose a soft sharing DA architecture with squared Maximum Mean Discrepancy (MMD) loss function. The source and target have a similar architecture, consisting of the hybrid CNN-GRU used for epilepsy, but their parameters are modeled with a non-linear relation. For generalization, we augmented the DA architecture with representation learning. We conducted a comprehensive set of experiments for DA with different scenarios of data shifts between source and target domains and showed where hard parameter sharing approach fails. We evaluated the solutions with three cases of DA in the context of activity recognition (AR). The input to the prediction model is multivariate time series data from wearable sensors on a smartphone and a smartwatch. The output is a particular user activity. The first adaptation case captures the scenario where the source domain consists of labeled activities for a group of users, and the target domain is a new user. The second scenario consists of the case where the source domain consists of labeled activities with data collected from one set of devices, and the target domain is a subset of the devices. The third scenario combines the first two cases, and the target domain has a new user and a new set of devices. Compared to the state-of-the-art, the results showed superior improvements up to 8% on average measured in weighted F1-score and reduction in variations of 3.5x on average.