LOW COMPLEXITY DATA PROCESSING AND LEARNING OVER SINGLE-AGENTS AND DISTRIBUTED NETWORKS

Abstract

An enormous amount of data is generated every second across a wide range of sectors around the globe. Learning from the available data for given use cases requires full data access, which can be challenging due to privacy considerations in addition to communications, storage and computational constraints. For these reasons, it is highly attractive to enable devices to process and learn from data locally to train a global model while exchanging only selected parameters with other devices over the network, and without the need to share and store the complete data set. This reduces the risk of privacy leakage and the communication load over the network. However, it transfers the computational burden to the end devices or local agents, which normally have energy and processing speed limitations. This PhD thesis deals with designing low complexity and effective algorithms for data processing, optimization, and learning over both single agents and networks. The thesis work is divided into three main parts. In the first part, we propose two methods for learning over networks; the first one addresses the problem of learning over heterogeneous networks and, in particular, how collaboration among heterogeneous agents can improve the overall learning performance, and the second one presents a novel low complexity approximation method for the gradient descent algorithm. Theoretical proofs of the convergence, complexity analysis, and performance results are provided and analyzed to demonstrate effectiveness and generate insights. In the second part, we present an efficient two-step design methodology for low complexity finite-impulse response (FIR) filters. In the first step, the filter design is formulated as an optimization problem to find the minimum number of coefficients. In the second step, a mapping approach is proposed to compensate for the mean square error (MSE) that results from the approximation in the first step. Simulation results show that the designed filter has flat response in the passband, a narrow transition band, and high attenuation in the stopband. The proposed design leads to a better performance-computational complexity tradeoff compared to other state-of-the-art digital filter design methods. Finally, in the third part, a dynamic compression algorithm for EEG biomedical data is proposed. The algorithm applies a sequence of compression/decompression operations in order to find an optimized lossless/lossy compression combination that provides high compression ratio while preserving the signal integrity. Performance evaluation results on real datasets demonstrate an effective compression performance while maintaining a distortion level below a target threshold and low computational overhead.