A hybrid game theory and reinforcement learning approach for cyber-physical systems security

Abstract

Cyber-Physical Systems (CPS) like smart cities and industries 4.0 including nuclear power plants, oil and gas pipelines, electric power grids, railways, and other Critical Infrastructures (CI) are monitored and controlled by Supervisory Control and Data Acquisition (SCADA) systems that use advanced computing, sensors, control systems, and communication network. At first, CPSs were protected and secured by isolation. However, with recent industrial technology advances, the increased connectivity of CPSs and SCADA systems to enterprise networks has uncovered them to new cybersecurity threats and made them a primary target for cyber-attacks with the potential of causing catastrophic economic, social, and environmental damage. This thesis work proposes two complementary cybersecurity risk assessment approaches to evaluate and assess cybersecurity for CPS networks extensively. First, we propose a game-theoretical model for cybersecurity in Industrial Control System (ICS) using Monte Carlo simulations to evaluate the payoffs, given different variants of randomness, selected strategies, budget spending, and look-ahead. Second, we propose a refined approach to frame CPS security in two different levels, strategic and battlefield, by meeting ideas from both game theory and Multi-Agent Reinforcement Learning (MARL). The strategic level is modeled as imperfect information, extensive form game. Here, the human administrator and the virus author decide on the strategies of defense and attack, respectively. At the battlefield level, strategies are implemented by machine learning agents that derive optimal policies for run-time decisions. The outcomes of these policies manifest as the utility at a higher level, where the aim is to reach a Nash Equilibrium (NE) in favor of the defender. A framework is implemented to simulate the scenario of a virus spreading in a realistic CPS network. Promising results show that the defender can learn optimal policies to counter viruses that could be equipped with Artificial Intelligence (AI)