完整搜尋頁面 →

軟 Dice 損失的梯度

選讀・DLCV 04・影像分割:語意、實例與全景分割 的補充資料(第 3/4 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 4

  1. 堆疊空洞卷積中的網格效應(gridding)
  2. 密集 CRF 後處理
  3. 軟 Dice 損失的梯度
  4. Boundary IoU 詳解

論文導讀 · 2

  1. Panoptic Segmentation:論文導讀
  2. Mask2Former:論文導讀

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

這則補充推導軟 Dice 係數的導數,並說明它不尋常的行為。

導數。 使用總和形式 D=2I/SD = 2I / S,其中 I=∑ipigiI = \sum_i p_i g_i、S=∑ipi+∑igiS = \sum_i p_i + \sum_i g_i,pi∈[0,1]p_i \in [0, 1] 是預測機率,gi∈{0,1}g_i \in \{0, 1\} 是標籤。由商的微分法則,對像素 jj,

∂D∂pj=2gjS−2IS2=2 (gjS−I)S2,\frac{\partial D}{\partial p_j} = \frac{2 g_j S - 2 I}{S^2} = \frac{2\,(g_j S - I)}{S^2},

因為 ∂I/∂pj=gj\partial I / \partial p_j = g_j 且 ∂S/∂pj=1\partial S / \partial p_j = 1。(Milletari 等人 [15] 給出了他們平方分母版本的類似式子。)

它說了什麼。

  • 每個背景像素(gj=0g_j = 0)對機率的導數都是 ∂D/∂pj=−2I/S2\partial D/\partial p_j = -2I/S^2,與目前機率無關。最小化 1−D1-D 因此會降低這些機率,但對機率的導數相同,不代表對網路 logits 的導數也相同。
  • 每個前景像素得到同樣的正梯度 2(S−I)/S22(S - I)/S^2。
  • 推力的大小取決於全域的總和。當預測與真實遮罩都很小時,SS 很小,梯度就很大;這就是 Dice 如何「聽見」小物體,也是它在訓練初期可能不穩定的原因。

比較導數時,必須使用相同的變數。 對獨立二元 sigmoid,令 logit 為 zjz_j,且 pj=σ(zj)p_j=\sigma(z_j),Dice 損失滿足 ∂(1−D)/∂zj=−(∂D/∂pj)pj(1−pj)\partial(1-D)/\partial z_j=-(\partial D/\partial p_j)p_j(1-p_j)。二元交叉熵對 logit 的導數是 pj−gjp_j-g_j,對機率的導數卻是 (pj−gj)/(pj(1−pj))(p_j-g_j)/(p_j(1-p_j))。多類別 softmax 必須使用會耦合各類別通道的完整 Jacobian,不能只乘上獨立 sigmoid 的係數。

下面的程式用有限差分檢查對機率的導數。

程式碼 · Python · 17 行
import numpy as np

rng = np.random.default_rng(1)
g = (rng.random(200) < 0.05).astype(float)       # 200 pixels, about 5% foreground
p = rng.uniform(0.05, 0.6, 200)                  # some soft prediction

def dice(p, g):
    return 2 * (p * g).sum() / (p.sum() + g.sum())

I, S = (p * g).sum(), p.sum() + g.sum()
grad = 2 * (g * S - I) / S ** 2                  # analytic dD/dp_j
eps = 1e-6
num = np.array([(dice(p + eps * np.eye(200)[j], g) - dice(p - eps * np.eye(200)[j], g)) / (2 * eps)
                for j in range(200)])
print("max |analytic - numeric|:", f"{np.abs(grad - num).max():.1e}")
print("distinct gradient values on background pixels:", np.unique(grad[g == 0].round(12)).size)
print(f"foreground gradient {grad[g == 1][0]:+.5f}, background gradient {grad[g == 0][0]:+.5f}")

輸出

max |analytic - numeric|: 9.0e-12
distinct gradient values on background pixels: 1
foreground gradient +0.02793, background gradient -0.00099

邊界情況。 如果真實標籤是空的(g=0g = 0),則 I=0I = 0,對任何非零的預測都有 D=0D = 0,所以即使預測近乎完美,損失仍是 1;如果兩者都是零,比值沒有定義。實作上會在分子與分母加上平滑常數、對整個批次而不是每張影像計算總和,或是加上交叉熵,讓梯度不會只依賴 Dice 比值。在上述獨立 sigmoid 情況中,額外的 pj(1−pj)p_j(1-p_j) 係數可能使飽和像素學得很慢,即使對機率的導數並非零。

參考文獻

  1. S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz and D. Terzopoulos, “Image segmentation using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, 2022. doi · arXiv
  2. X. Li, H. Ding, H. Yuan, W. Zhang, J. Pang, G. Cheng, K. Chen, Z. Liu and C. C. Loy, “Transformer-based visual segmentation: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, 2024. doi · arXiv
  3. T. Zhou, W. Xia, F. Zhang, B. Chang, W. Wang, Y. Yuan, E. Konukoglu and D. Cremers, “Image segmentation in foundation model era: A survey,” arXiv:2408.12957, 2024. arXiv
  4. A. Kirillov, K. He, R. Girshick, C. Rother and P. Dollár, “Panoptic segmentation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
  5. J. Long, E. Shelhamer and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. doi · arXiv
  6. O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), LNCS vol. 9351, 2015. doi
  7. L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proc. European Conference on Computer Vision (ECCV), 2018. doi · arXiv
  8. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, 2018. doi · arXiv
  9. F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” in Proc. International Conference on Learning Representations (ICLR), 2016. arXiv
  10. L.-C. Chen, G. Papandreou, F. Schroff and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” arXiv:1706.05587, 2017. arXiv
  11. P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou and G. Cottrell, “Understanding convolution for semantic segmentation,” in Proc. IEEE Winter Conference on Applications of Computer Vision (WACV), 2018. doi · arXiv
  12. H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia, “Pyramid scene parsing network,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. doi · arXiv
  13. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. International Conference on Learning Representations (ICLR), 2021. arXiv
  14. E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” in Advances in Neural Information Processing Systems (NeurIPS), 2021. NeurIPS · arXiv
  15. F. Milletari, N. Navab and S.-A. Ahmadi, “V-Net: Fully convolutional neural networks for volumetric medical image segmentation,” in Proc. Fourth International Conference on 3D Vision (3DV), 2016. doi · arXiv
  16. T.-Y. Lin, P. Goyal, R. Girshick, K. He and P. Dollár, “Focal loss for dense object detection,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi
  17. B. Cheng, A. G. Schwing and A. Kirillov, “Per-pixel classification is not all you need for semantic segmentation,” in Advances in Neural Information Processing Systems (NeurIPS), 2021. NeurIPS · arXiv
  18. M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth and B. Schiele, “The Cityscapes dataset for semantic urban scene understanding,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. doi · arXiv
  19. B. Cheng, R. Girshick, P. Dollár, A. C. Berg and A. Kirillov, “Boundary IoU: Improving object-centric image segmentation evaluation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. doi · arXiv
  20. K. He, G. Gkioxari, P. Dollár and R. Girshick, “Mask R-CNN,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
  21. S. Ren, K. He, R. Girshick and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, 2017. doi
  22. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, “Feature pyramid networks for object detection,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. doi
  23. A. Kirillov, Y. Wu, K. He and R. Girshick, “PointRend: Image segmentation as rendering,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. doi · arXiv
  24. A. Kirillov, R. Girshick, K. He and P. Dollár, “Panoptic feature pyramid networks,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
  25. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. European Conference on Computer Vision (ECCV), 2020. doi · arXiv
  26. B. Cheng, I. Misra, A. G. Schwing, A. Kirillov and R. Girdhar, “Masked-attention mask transformer for universal image segmentation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. doi · arXiv
  27. J. Jain, J. Li, M. Chiu, A. Hassani, N. Orlov and H. Shi, “OneFormer: One transformer to rule universal image segmentation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. doi · arXiv
  28. T. Kerssies, N. Cavagnero, A. Hermans, N. Norouzi, G. Averta, B. Leibe, G. Dubbelman and D. de Geus, “Your ViT is secretly an image segmentation model,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. CVF · arXiv
  29. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson et al., “Segment anything,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2023. doi · arXiv
  30. N. Ravi, V. Gabeur, Y.-T. Hu, R. Hu, C. Ryali, T. Ma et al., “SAM 2: Segment anything in images and videos,” in Proc. International Conference on Learning Representations (ICLR), 2025. OpenReview · arXiv
  31. N. Carion, L. Gustafson, Y.-T. Hu, S. Debnath, R. Hu, D. Suris et al., “SAM 3: Segment anything with concepts,” in Proc. International Conference on Learning Representations (ICLR), 2026. OpenReview · arXiv
  32. A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal et al., “Learning transferable visual models from natural language supervision,” in Proc. International Conference on Machine Learning (ICML), PMLR 139, 2021. PMLR
  33. B. Li, K. Q. Weinberger, S. Belongie, V. Koltun and R. Ranftl, “Language-driven semantic segmentation,” in Proc. International Conference on Learning Representations (ICLR), 2022. OpenReview · arXiv
  34. S. Cho, H. Shin, S. Hong, A. Arnab, P. H. Seo and S. Kim, “CAT-Seg: Cost aggregation for open-vocabulary semantic segmentation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. doi · arXiv
  35. C. Zhu and L. Chen, “A survey on open-vocabulary detection and segmentation: Past, present, and future,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, 2024. doi
  36. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn and A. Zisserman, “The PASCAL Visual Object Classes (VOC) challenge,” International Journal of Computer Vision, vol. 88, no. 2, 2010. doi
  37. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso and A. Torralba, “Scene parsing through ADE20K dataset,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. doi · CVF
  38. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan et al., “Microsoft COCO: Common objects in context,” in Proc. European Conference on Computer Vision (ECCV), 2014. doi · arXiv
  39. H. Caesar, J. Uijlings and V. Ferrari, “COCO-Stuff: Thing and stuff classes in context,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. doi · arXiv
  40. G. Neuhold, T. Ollmann, S. Rota Bulò and P. Kontschieder, “The Mapillary Vistas dataset for semantic understanding of street scenes,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi
  41. P. Krähenbühl and V. Koltun, “Efficient inference in fully connected CRFs with Gaussian edge potentials,” in Advances in Neural Information Processing Systems (NeurIPS), 2011. NeurIPS · arXiv