mHEALTH System for Dermatology Diseases in Refugee Settlements Using Multi-Modal Classification

Abstract

While conflicts and wars are continuously erupting throughout the world, the dispersion of refugees from their war-afflicted countries to neighboring states is continuously increasing to an extent that these hosting states become incapable of meeting the refugees' basic needs, such as shelter, water, education, and most importantly, healthcare. In particular, the hardships that refugees have been facing due to the lack of basic medical services have motivated researchers to develop technological automated solutions to address existing challenges and enhance healthcare support.

The focus of this thesis is on developing a mobile health system for diagnosing certain skin diseases in an automated and accurate manner. The proposed system leverages recent advances in machine learning, in particular deep learning algorithms that are applied to different data modalities. The system's architecture includes a user question-answer component to retrieve user-related background and health information, followed by an embedding model that is used to learn representations for uploaded images capturing the affected areas of their skin.

Finally, a machine learning classifier is trained using the features extracted from both the questionnaire and image modalities to accurately predict the type of skin disease, providing preliminary input to remote medical experts for further evaluation and treatment. Testing and evaluation are performed using various real data sets, and the obtained results demonstrate the overall effectiveness of the proposed approach.