How to Cite
Abstract
In this paper, we aim to investigate and defend autonomous vehicle (AV) control policies, which were designed by reinforcement learning (RL) agents, against adversarial attacks. Ensuring the safety and security of RL-based control policies becomes even more critical, especially if they are designed for high-risk tasks, such as AVs, because adversaries could exploit model vulnerabilities and real-time AV sensor data to cause adversarial perturbations to the control policy. These adversarially generated perturbations enable the adversaries to manipulate the RL-based AV control policy at test time and can cause unsafe actions such as accidents, privacy violations, and financial losses.
In this paper, we propose an adversarial defense mechanism based on robustifying an artificial agent's policy over training time and a large-scale ensemble policy that further improves robustness. Specifically, in both defenses, novel augmentation-based reward shaping mechanisms are proposed to improve the performance and stability of the artificial agent during various stages of training and testing. We evaluate the performance of our defense mechanisms in various real-world adversarial environments and demonstrate the superiority of the proposed defense mechanisms over the state-of-the-art in the context of autonomous vehicle control using MuJoCo.
References
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, "The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results," in Proceedings of the 5th International Workshop on Visual Object Classification, 2007, pp. 1-34.
K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv:1409.1556, 2014.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," in Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, 2012, pp. 1097-1105.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep Learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," in Advances in Neural Information Processing Systems 28, 2015, pp. 91-99.
A. Krizhevsky, "Learning Multiple Layers of Features from Tiny Images," Master's thesis, University of Toronto, 2009.
O. Russakovsky et al., "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, vol. 115, no. 3, pp. 211-252, 2015.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
M. D. Zeiler and R. Fergus, "Visualizing and Understanding Convolutional Networks," in European Conference on Computer Vision, 2014, pp. 818-833.
Tatineni, Sumanth. "INTEGRATING AI, BLOCKCHAIN AND CLOUD TECHNOLOGIES FOR DATA MANAGEMENT IN HEALTHCARE." Journal of Computer Engineering and Technology (JCET) 5.01 (2022).
Vemori, Vamsi. "Evolutionary Landscape of Battery Technology and its Impact on Smart Traffic Management Systems for Electric Vehicles in Urban Environments: A Critical Analysis." Advances in Deep Learning Techniques 1.1 (2021): 23-57.
Shaik, Mahammad, and Ashok Kumar Reddy Sadhu. "Unveiling the Synergistic Potential: Integrating Biometric Authentication with Blockchain Technology for Secure Identity and Access Management Systems." Journal of Artificial Intelligence Research and Applications 2.1 (2022): 11-34.
K. He, X. Zhang, S. Ren, and J. Sun, "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification," in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1026-1034.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
M. D. Zeiler, "ADADELTA: An Adaptive Learning Rate Method," arXiv:1212.5701, 2012.
A. Krizhevsky and G. Hinton, "Learning Multiple Layers of Features from Tiny Images," Technical Report, University of Toronto, 2009.
I. Goodfellow et al., "Generative Adversarial Nets," in Advances in Neural Information Processing Systems 27, 2014, pp. 2672-2680.
K. He, "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904-1916, 2015.
D. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," arXiv:1412.6980, 2014.
A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-Scale Video Classification with Convolutional Neural Networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1725-1732.
D. Silver et al., "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm," arXiv:1712.01815, 2017.