Distributed Logistic Classifiers in Semi-Supervised Settings

Abstract

In network semi-supervised learning problems, only a subset of the network nodes is able to access the data labeling. This thesis formulates a decentralized optimization problem where agents represent classifiers that may observe different numbers and types of features, and hence have individual decision rules to estimate, subject to the condition that neighboring agents are more likely to have similar labels. To promote such relationships, we propose to add to the individual logistic regression costs a graph regularization term that allows to penalize the differences between the labels at neighboring agents. Two regularization terms are investigated: a sparsity promoting regularizer and a smoothness promoting regularizer. Streaming data is assumed, and therefore, the \emph{stochastic} (sub-)gradient descent method is used to solve the regularized problem. We provide some important assumptions and conditions that guarantee the stability and convergence of the proposed algorithm in the mean-square-error sense. Simulation results show that collaboration among neighboring agents, which is promoted through the added regularization term, can lead to better classification results by decreasing the probability of error and by improving the convergence rate. Those results are promising in semi-supervised settings where some agents do not have access to labeled data points due to cost or privacy reasons, and in applications with limited amount of data.