Improving robustness of traffic sign recognition against adversarial patches using vision transformers and feature purificatio
Article Sidebar
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Main Article Content
Abstract
Traffic sign recognition (TSR) is a safety-critical component of driver-assistance and autonomous driving systems, yet it remains vulnerable to adversarial patch attacks that are physically realizable and effective under common viewing conditions. This paper investigates a robustness-oriented TSR approach that combines Vision Transformers (ViTs) with feature purification to suppress patch-induced feature contamination while preserving sign-relevant semantics. The method estimates patch influence at the token level and applies a lightweight purification mechanism to down-weight suspicious tokens and reinforce stable contextual cues prior to classification. A training protocol based on diverse synthetic patch augmentation and transformation-aware consistency regularization is described to improve generalization to unseen patch patterns and real-world transformations. The proposed evaluation reports clean accuracy, robust accuracy, and attack success rate under targeted and untargeted patch attacks with realistic geometric and photometric variations. The study aims to provide a practical defense layer that improves recognition reliability without redesigning existing TSR pipelines.
Article Details
References
T. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, “Adversarial Patch,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4724–4732.
https://doi.org/10.1109/CVPR.2017.500 DOI: https://doi.org/10.1109/CVPR.2017.500
K. Eykholt, I. Evtimov, E. Fernandes, et al., “Robust Physical-World Attacks on Deep Learning Visual Classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 1625–1634.
https://doi.org/10.1109/CVPR.2018.00175 DOI: https://doi.org/10.1109/CVPR.2018.00175
A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale,” in International Conference on Learning Representations (ICLR), 2021.
https://openreview.net/forum?id=YicbFdNTTy
A. Bhojanapalli, A. Chakrabarti, A. Garg, et al., “Understanding Robustness of Transformers for Image Classification,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 10231–10241. DOI: https://doi.org/10.1109/ICCV48922.2021.01007
https://doi.org/10.1109/ICCV48922.2021.01006 DOI: https://doi.org/10.1109/ICCV48922.2021.01006
H. Salman, A. Ilyas, L. Engstrom, et al., “Do Adversarially Robust ImageNet Models Transfer Better?” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020.
https://proceedings.neurips.cc/paper/2020/hash/24357dd085d2b1b82e55ef2b9a7d9c6c-Abstract.html
Y. Carmon, A. Raghunathan, L. Schmidt, J. Duchi, and P. Liang, “Unlabeled Data Improves Adversarial Robustness,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019.
https://proceedings.neurips.cc/paper/2019/hash/6def6e0d64a86c27e09c98752f6c6b5a-Abstract.html
J. Wang, R. Zhang, C. Xie, et al., “Adversarial Purification with Score-Based Generative Models,” in Proceedings of the International Conference on Machine Learning (ICML), 2022.
https://proceedings.mlr.press/v162/wang22t.html
M. Naseer, K. Ranasinghe, S. Khan, et al., “On Improving Adversarial Robustness of Vision Transformers,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 225–241, 2023. DOI: https://doi.org/10.1109/TPAMI.2022.3207917
https://doi.org/10.1109/TPAMI.2022.3148920
S. Wang, Y. Sun, Z. Chen, and X. Wang, “PatchGuard++: Efficient and Robust Defense Against Adversarial Patches,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 6, pp. 7341–7356, 2023.
https://doi.org/10.1109/TPAMI.2022.3209404
Y. Li, Y. Bai, Y. Jiang, et al., “Improving Robustness of Vision Transformers via Token Refinement,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
https://doi.org/10.1109/CVPR52729.2023.01241 DOI: https://doi.org/10.1109/CVPR52729.2023.01241
A. Singh, R. Bala, and S. R. Dubey, “Robust Traffic Sign Recognition Using Deep Learning: A Survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 4, pp. 3812–3830, 2023.