Multi-modal Representation Learning with Contrastive and Prototype Alignment for Alzheimer’s Classification

Abstract

Alzheimer’s disease (AD) classification remains a challenging task due to the complexity of the disease and the limitations of single-modality learning approaches. In this thesis, we propose a multi-modal machine learning framework that integrates structural MRI data with clinical tabular information to improve diagnostic performance while maintaining clinical feasibility. The approach leverages supervised contrastive learning to learn discriminative and semantically meaningful representations within and across modalities, enhancing class separability for AD staging.

To address the challenges of data heterogeneity and limited generalization in multi-modal settings, we introduce a supervised contrastive pretraining stage that aligns imaging and tabular representations in a shared embedding space. Experimental results demonstrate that the proposed framework achieves performance comparable to state-of-the-art methods while relying only on widely available modalities.

Building upon this centralized framework, we further propose a multi-modal federated learning approach that integrates prototype-based supervised contrastive learning. In this setting, each client learns local representations while sharing both model parameters and class-wise prototypes with a central server. The server aggregates these prototypes across clients using weighted averaging and normalization to construct global class representations.

These global prototypes are then used to guide both intra-modal representation learning and cross-modal alignment between MRI and tabular data. This approach improves representation consistency across clients, mitigates client drift, and enhances performance under non-IID data distributions.

Overall, this work contributes to a multi-modal and federated learning framework for the classification of Alzheimer’s disease, addressing key challenges in representation learning, data heterogeneity, and robust learning in distributed medical environments.