Defense Mechanisms for Mitigating Adversarial Attacks Against Deep Learning-Based Traffic Signs Recognition of Autonomous Driving Cars

Abstract

Artificial Intelligence (AI) techniques, particularly machine learning and deep learning, have gained significant attention for their ability to solve complex problems. Among these, Deep Neural Networks (DNNs) have demonstrated remarkable success in handling nonlinear problems due to their capacity to process vast amounts of data during training. However, the widespread deployment of DNNs in critical applications has raised concerns regarding their security and robustness. Notably, DNNs are vulnerable to adversarial attacks—carefully crafted perturbations that manipulate input data to deceive the model into making incorrect predictions.

One of the most critical applications of DNNs is in autonomous vehicle systems, which rely on deep learning models for tasks such as object detection, localization, navigation, and trajectory planning. Despite their effectiveness, adversarial attacks on these models can lead to erroneous decisions with potentially catastrophic consequences. In particular, compromising the traffic sign recognition system of an autonomous vehicle can severely impact its decision-making process, posing serious safety risks.

This dissertation proposes three novel defense mechanisms to mitigate adversarial attacks on DNN-based traffic sign recognition systems in autonomous vehicles. The first approach enhances input robustness by augmenting acquired data with descriptive metadata, including a segmented and an inverted version of the input. Beside the main model, another model called a verifier is employed. By comparing predictions across multiple versions of the inputs, this method effectively detects adversarial manipulations. The second approach applies image transformations, such as splitting and flipping, to disrupt adversarial perturbations while preserving the integrity of clean inputs. The third defense mechanism leverages Siamese Neural Networks for similarity learning. Unlike traditional applications of Siamese Networks for object recognition, this work utilizes them for adversarial detection. The main model predicts a class to the input, then two random clean samples are drawn from the same class for similarity comparison with the input. The Siamese embedding representations of the input and the random samples are compared using the cosine similarity measure. If the input deviates significantly in feature space, it is flagged as adversarial.

Additionally, to enhance detection performance across various attack types, all proposed methods incorporate anomaly detection using a One-Class Support Vector Machine (OC-SVM). By exploring diverse defense strategies, this research aims to develop robust and generalizable mechanisms to secure DNN-based traffic sign recognition systems in autonomous vehicles, as well as other critical AI-driven applications, against adversarial threats.