Reinforcement Learning Based Scheme For On-Demand Vehicular Fog Formation and Micro Services Placement

Abstract

The high need of real-time vehicular applications for self-driving cars to maintain service availability and reachability, and to process huge amount of generated data within a small amount of time, rise the need to improve the vehicular network infrastructure. Fog computing has been introduced to reduce the amount of data sent to cloud by bringing processing power near the edge and reducing latency. In this paper, we overcome the aforementioned limitations by taking advantage of the evolvement of On-Board Units, Reinforcement Learning, Kubeadm Clustering, Docker Containerization, Istio service mesh, and micro-services technologies. We propose in our scheme (1) a service mesh architecture that manages communication between multiple microservices across different clusters and tackle inter-service communication, (2) a Reinforcement Learning model deployed on road side units to predict on-demand placement of microservices, and (3) Reformulating the vehicular container placement (VCP) problem as an assignment problem to map microservices to vehicles. Experiments and simulations show that our method is more efficient in deploying microservices using the reinforcement learning model than other current strategies in the literature. Given that only needed microservices are deployed in limited resource cluster, mesh network shows an improvement in the deployment time and inter-service communication across clusters, and the usage of Hungarianalgorithm in solving the VCP improves Quality of Service (QoS).