MetaDial: A Meta-learning Approach for Dialogue Generation in Arabic Language

Abstract

Dialogue generation is the automatic generation of a text response, given a post by a user. The advancements in deep learning models have made developing conversational systems not only possible, but also effective and helpful in many applications spanning a variety of domains. Nevertheless, work on Arabic Conversational bots is still limited due to various challenges including the language rich morphology, huge vocabulary, and the scarcity of data resources. Although meta-learning has been introduced before in the natural language processing (NLP) realm and showed significant improvements in many tasks, it has rarely been used in natural language generation (NLG) tasks and never in Arabic NLG. In this thesis, we propose a meta-learning approach for Arabic Dialogue generation for fast adaptation on low resource domains. We start by using existing pre-trained models; we then meta-learn the initial parameters on high resource dataset before fine-tuning the parameters on the target tasks. We prove that the proposed model that employs meta-learning techniques improves generalization and enables fast adaptation of the transformer model on low-resource NLG tasks. We report gains in the BLEU-4 in improvements in Semantic textual Similarity (STS) metrics in comparison with the existing state-of-the-art approach. We also do a further study on the effectiveness of the meta-learning algorithms on the response generation of the models.