Athalye 等人:Obfuscated Gradients Give a False Sense of Security(ICML 2018)
選讀・DLCV 14・穩健性:分布偏移、對抗樣本與異常 的論文導讀(第 2/3 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
挑戰
許多防禦報告了對白箱迭代攻擊的高穩健性。它們真的穩健,還是只是讓攻擊者的梯度變得無用 [29]?

方法
找出機制,也就是以混淆梯度形式出現的梯度遮蔽,並建造不依賴被遮蔽梯度的攻擊。論文命名了三種類型(破碎、隨機,以及消失或爆炸梯度),列出能揭露它們的行為(正文中的警訊),並為每種類型給出反制攻擊:BPDA、EOT 與重新參數化。
結果
在針對 ICLR 2018 收錄的非認證白箱防禦所做的案例研究中,九個有七個依賴混淆梯度;新的攻擊在各論文原本考慮的威脅模型內,完全繞過了其中六個,部分繞過一個 [29]。兩個認證防禦與一個只主張黑箱的防禦,沒有放進案例研究。論文也強調一種不對稱:攻破一個防禦,一種攻擊就夠了,而防禦者必須證明威脅模型內沒有任何攻擊會成功。
判讀證據
先讀第 3 節(三種類型與它們的症狀)與第 4 節(攻擊),再讀表 1,它列出每個防禦的資料集、擾動界,以及在新攻擊下剩下的準確率。注意表格底部的兩個對抗訓練防禦:作者發現 Madry 等人的 PGD 訓練並不依賴混淆梯度。要檢查:(a) 每個攻擊是否保留原論文的威脅模型與 ,讓這些數字是公平的重新評估;(b) 逐一防禦的各節,它們顯示每個案例中 BPDA 的近似是怎麼選的;(c) 討論中的評估建議。
限制與問題
這項分析是針對一場會議上的防禦所做的案例研究;它顯示這些防禦會失敗,並不表示某一類的所有防禦都必然失敗。適應性攻擊需要付出努力與專業知識,這個缺口正是後來 AutoAttack 想在標準設定中補上的 [30]。
結論
這就是本篇要求先經過白箱、適應性與集成攻擊,才相信任何穩健性主張的原因,也是練習 2 所用警訊的來源。
沿調適梯度追蹤計算
BPDA 保留防禦真正的前向轉換 ,反向則改用可微分近似 。若 是到達轉換輸出的損失梯度,近似輸入梯度為 , 是 的 Jacobian。例如前向仍執行四捨五入,反向以恆等映射提供導數;產生候選後,必須在真正的取整模型評估,不能只測替代模型 [29]。
若轉換 帶隨機性,EOT 以抽樣估計期望損失梯度:
為受防禦預測器, 是攻擊目標損失 使用的標籤, 為樣本數, 為抽到的轉換。交換微分、期望需要對應正則條件,不可微轉換可能還要反向近似。增加攻擊計算改變經驗檢查的可靠度,不改變允許的擾動集合 [29]。
參考文獻
- D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt and D. Song, “Natural Adversarial Examples,” CVPR, 2021. CVF
- P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani et al., “WILDS: A Benchmark of in-the-Wild Distribution Shifts,” ICML, PMLR 139, 2021. PMLR
- J. Yang, K. Zhou, Y. Li and Z. Liu, “Generalized Out-of-Distribution Detection: A Survey,” International Journal of Computer Vision, vol. 132, no. 12, pp. 5635–5662, 2024. doi · arXiv
- D. Hendrycks and T. Dietterich, “Benchmarking Neural Network Robustness to Common Corruptions and Perturbations,” ICLR, 2019. arXiv
- D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo et al., “The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization,” ICCV, 2021. CVF
- A. Barbu, D. Mayo, J. Alverio, W. Luo, C. Wang, D. Gutfreund, J. Tenenbaum and B. Katz, “ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models,” NeurIPS, 2019. proceedings
- B. Recht, R. Roelofs, L. Schmidt and V. Shankar, “Do ImageNet Classifiers Generalize to ImageNet?,” ICML, PMLR 97, 2019. PMLR
- R. Taori, A. Dave, V. Shankar, N. Carlini, B. Recht and L. Schmidt, “Measuring Robustness to Natural Distribution Shifts in Image Classification,” NeurIPS, 2020. proceedings
- J. Miller, R. Taori, A. Raghunathan, S. Sagawa, P. W. Koh, V. Shankar, P. Liang, Y. Carmon and L. Schmidt, “Accuracy on the Line: on the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization,” ICML, PMLR 139, 2021. PMLR
- D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer and B. Lakshminarayanan, “AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty,” ICLR, 2020. arXiv
- R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann and W. Brendel, “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness,” ICLR, 2019. arXiv
- A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, PMLR 139, 2021. PMLR
- A. Fang, G. Ilharco, M. Wortsman, Y. Wan, V. Shankar, A. Dave and L. Schmidt, “Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP),” ICML, PMLR 162, 2022. PMLR
- M. Wortsman, G. Ilharco, J. W. Kim, M. Li, S. Kornblith, R. Roelofs et al., “Robust Fine-Tuning of Zero-Shot Models,” CVPR, 2022. CVF
- S. Schneider, E. Rusak, L. Eck, O. Bringmann, W. Brendel and M. Bethge, “Improving robustness against common corruptions by covariate shift adaptation,” NeurIPS, 2020. proceedings
- D. Wang, E. Shelhamer, S. Liu, B. Olshausen and T. Darrell, “Tent: Fully Test-Time Adaptation by Entropy Minimization,” ICLR, 2021. arXiv
- Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand and V. Lempitsky, “Domain-Adversarial Training of Neural Networks,” Journal of Machine Learning Research, vol. 17, no. 59, pp. 1–35, 2016. JMLR
- K. Zhou, Z. Liu, Y. Qiao, T. Xiang and C. C. Loy, “Domain Generalization: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022. doi · arXiv
- I. Gulrajani and D. Lopez-Paz, “In Search of Lost Domain Generalization,” ICLR, 2021. arXiv
- Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. V. Dillon, B. Lakshminarayanan and J. Snoek, “Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift,” NeurIPS, 2019. proceedings
- C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow and R. Fergus, “Intriguing properties of neural networks,” ICLR, 2014. arXiv
- I. J. Goodfellow, J. Shlens and C. Szegedy, “Explaining and Harnessing Adversarial Examples,” ICLR, 2015. arXiv
- A. Madry, A. Makelov, L. Schmidt, D. Tsipras and A. Vladu, “Towards Deep Learning Models Resistant to Adversarial Attacks,” ICLR, 2018. arXiv
- F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal and M. Hein, “RobustBench: a standardized adversarial robustness benchmark,” NeurIPS Datasets and Benchmarks Track, 2021. proceedings · leaderboard
- N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik and A. Swami, “Practical Black-Box Attacks against Machine Learning,” ACM Asia Conference on Computer and Communications Security (AsiaCCS), pp. 506–519, 2017. doi
- N. Carlini and D. Wagner, “Towards Evaluating the Robustness of Neural Networks,” IEEE Symposium on Security and Privacy (SP), pp. 39–57, 2017. doi
- D. Tsipras, S. Santurkar, L. Engstrom, A. Turner and A. Madry, “Robustness May Be at Odds with Accuracy,” ICLR, 2019. arXiv
- H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. El Ghaoui and M. I. Jordan, “Theoretically Principled Trade-off between Robustness and Accuracy,” ICML, PMLR 97, 2019. PMLR
- A. Athalye, N. Carlini and D. Wagner, “Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples,” ICML, PMLR 80, 2018. PMLR
- F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” ICML, PMLR 119, 2020. PMLR
- J. Cohen, E. Rosenfeld and J. Z. Kolter, “Certified Adversarial Robustness via Randomized Smoothing,” ICML, PMLR 97, 2019. PMLR
- K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno and D. Song, “Robust Physical-World Attacks on Deep Learning Visual Classification,” CVPR, 2018. CVF
- T. B. Brown, D. Mané, A. Roy, M. Abadi and J. Gilmer, “Adversarial Patch,” arXiv:1712.09665, 2017. arXiv
- D. Hendrycks and K. Gimpel, “A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks,” ICLR, 2017. arXiv
- W. Liu, X. Wang, J. D. Owens and Y. Li, “Energy-based Out-of-distribution Detection,” NeurIPS, 2020. proceedings
- K. Lee, K. Lee, H. Lee and J. Shin, “A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks,” NeurIPS, 2018. proceedings
- Y. Sun, Y. Ming, X. Zhu and Y. Li, “Out-of-Distribution Detection with Deep Nearest Neighbors,” ICML, PMLR 162, 2022. PMLR
- D. Hendrycks, M. Mazeika and T. Dietterich, “Deep Anomaly Detection with Outlier Exposure,” ICLR, 2019. arXiv
- J. Zhang, J. Yang, P. Wang, H. Wang, Y. Lin, H. Zhang et al., “OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection,” arXiv:2306.09301, 2023. arXiv
- P. Bergmann, M. Fauser, D. Sattlegger and C. Steger, “MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection,” CVPR, 2019. CVF
- L. Heckler-Kram, J.-H. Neudeck, U. Scheler, R. König and C. Steger, “The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection,” International Journal of Computer Vision, vol. 134, no. 4, 2026. doi · arXiv
- K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox and P. Gehler, “Towards Total Recall in Industrial Anomaly Detection,” CVPR, 2022. CVF
- J. Bitterwolf, M. Müller and M. Hein, “In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation,” ICML, PMLR 202, 2023. PMLR
- J. Liang, R. He and T. Tan, “A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts,” International Journal of Computer Vision, vol. 133, no. 1, pp. 31–64, 2025. doi · arXiv
- N. Akhtar, A. Mian, N. Kardan and M. Shah, “Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey,” IEEE Access, vol. 9, pp. 155161–155196, 2021. doi · arXiv
- G. Pang, C. Shen, L. Cao and A. van den Hengel, “Deep Learning for Anomaly Detection: A Review,” ACM Computing Surveys, vol. 54, no. 2, pp. 1–38, 2021. doi · arXiv
- J. Liu, G. Xie, J. Wang, S. Li, C. Wang, F. Zheng and Y. Jin, “Deep Industrial Image Anomaly Detection: A Survey,” Machine Intelligence Research, vol. 21, no. 1, pp. 104–135, 2024. doi · arXiv
- Z. Wang, T. Pang, C. Du, M. Lin, W. Liu and S. Yan, “Better Diffusion Models Further Improve Adversarial Training,” ICML, PMLR 202, 2023. PMLR
- J. Yang, P. Wang, D. Zou, Z. Zhou, K. Ding, W. Peng et al., “OpenOOD: Benchmarking Generalized Out-of-Distribution Detection,” NeurIPS Datasets and Benchmarks Track, 2022. proceedings
- J. Jeong, Y. Zou, T. Kim, D. Zhang, A. Ravichandran and O. Dabeer, “WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation,” CVPR, 2023. CVF