ReviewModus: Text classification and sentiment prediction of unstructured reviews using a hybrid combination of machine learning and evaluation models

Abstract

While research interest on product and service evaluation from unstructured text reviews is increasing, investigating the effectiveness of predictive analytical models in this context is still under-explored. With the advancement in machine learning research, an opportunity exists to bridge this gap using a model-based product and service evaluation. We propose in this article ReviewModus, a text mining and processing framework that (1) relies on the model structure and its corresponding assessment questions to train a machine learning algorithm to predict the classification of reviews around the model dimensions; (2) predicts the sentiments within the reviews based on external review training datasets; and (3) transforms the extracted measures from the reviews for further analysis. Our approach is evaluated in the context of 11 e-government services where the performance of the framework is compared to the manual processing of unstructured reviews cross-checked by three independent evaluators. Our study shows promising classification results with a micro-average F-score reaching 85.16%, and a high sentiment prediction correlation (71.44%) with the manually performed sentiment assessment. © 2019 Elsevier Inc.