Full search page →

RoI pooling versus RoIAlign: sampling without rounding

Optional · supplementary note 3 of 5 for DLCV 03 · Object Detection: From R-CNN to DETR. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 5

  1. Classical detectors in detail: Viola–Jones, HOG and DPM
  2. Anchor shapes by k-means: clustering with an IoU distance
  3. RoI pooling versus RoIAlign: sampling without rounding
  4. The YOLO family, version by version
  5. DETR descendants in detail

Paper guides · 1

  1. Guide to the Modern view surveys

Deep Learning for Computer Vision

  1. 02 · Deep Image Classification
View the whole course tree →

Plain version. RoI pooling cuts a region out of the feature map with scissors that can only follow the grid lines, so the cut snaps to the nearest line. RoIAlign cuts exactly where the box is and reads the values between grid lines by blending the neighbouring cells.

Let the feature map FF have stride ss (one cell covers s×ss \times s image pixels), and let an RoI be given in image pixels. A pixel coordinate xx becomes the feature coordinate x/sx/s. Here cell jj covers [j,j+1)[j, j+1) and its value sits at its center j+12j + \tfrac12.

RoI pooling [9] rounds the RoI to whole cells, splits it into an H×WH \times W grid of bins, rounds the bin edges to whole cells too, and takes the maximum inside each bin. Every rounding moves a boundary by up to half a cell, which is s/2s/2 image pixels, and the error is different for every RoI [16].

RoIAlign [16] keeps every coordinate real-valued. Each bin gets a regular grid of sample points (the paper uses four regularly spaced points per bin; the snippet below uses two along xx), and each point is read by bilinear interpolation. For a point (x,y)(x, y) in feature coordinates, set u=x−12u = x - \tfrac12, v=y−12v = y - \tfrac12, j=⌊u⌋j = \lfloor u \rfloor, i=⌊v⌋i = \lfloor v \rfloor, a=u−ja = u - j and b=v−ib = v - i. Then

F(x,y)=(1−b)[(1−a)Fi,j+aFi,j+1]+b[(1−a)Fi+1,j+aFi+1,j+1],F(x, y) = (1 - b)\big[(1 - a) F_{i,j} + a F_{i,j+1}\big] + b\big[(1 - a) F_{i+1,j} + a F_{i+1,j+1}\big],

where Fi,jF_{i,j} is the feature value of the cell in row ii and column jj, aa and bb are the fractional offsets of the point from cell center (j,i)(j, i), and the four terms weight the four surrounding cells. The bin’s output is the maximum or the average of its sample points; the paper uses the average. The paper reports that results are not sensitive to the exact sample positions or their number, as long as nothing is rounded [16]. Gradients reach the four neighbours of every sample point, so the layer trains like any other.

The snippet uses a feature map whose value equals the xx coordinate of each cell center, so a correct pooling should return the xx coordinate of the middle of each bin. Both methods use averages here, to isolate position errors from the effect of taking a maximum. The RoI keeps its width of 64 pixels and moves right by 4 pixels per row.

Code · Python · 36 lines
import numpy as np

STRIDE = 16
F = np.tile(np.arange(8) + 0.5, (8, 1))   # F[i, j] = x-coordinate of the centre of cell j (in cells)

def roi_pool_avg(F, x1, x2, bins=2):
    """Quantized pooling along x (average). Returns the mean cell-centre read by each bin."""
    s, e = int(round(x1 / STRIDE)), int(round(x2 / STRIDE))     # quantize the RoI
    edges = s + (e - s) * np.arange(bins + 1) / bins
    out = []
    for a, b in zip(edges[:-1], edges[1:]):
        lo, hi = int(np.floor(a)), int(np.ceil(b))              # quantize the bin edges
        out.append(F[0, lo:hi].mean())
    return np.array(out)

def bilinear(F, x, y=0.0):
    """Sample F at a continuous point (x, y); cell j has its centre at x = j + 0.5."""
    u, v = x - 0.5, y - 0.5
    j0, i0 = int(np.floor(u)), int(np.floor(v))
    du, dv = u - j0, v - i0
    j0c, j1c = np.clip([j0, j0 + 1], 0, F.shape[1] - 1)
    i0c, i1c = np.clip([i0, i0 + 1], 0, F.shape[0] - 1)
    return ((1 - dv) * ((1 - du) * F[i0c, j0c] + du * F[i0c, j1c])
            + dv * ((1 - du) * F[i1c, j0c] + du * F[i1c, j1c]))

def roi_align_avg(F, x1, x2, bins=2, n=2):
    """RoIAlign along x: n sample points per bin on a regular grid, no rounding, then average."""
    a, b = x1 / STRIDE, x2 / STRIDE
    w = (b - a) / bins
    return np.array([np.mean([bilinear(F, a + k * w + (t + 0.5) * w / n) for t in range(n)])
                     for k in range(bins)])

for x1, x2 in [(22, 86), (26, 90), (30, 94)]:                    # same width, moved right by 4 px each step
    true_c = (np.arange(2) + 0.5) / 2 * (x2 - x1) / STRIDE + x1 / STRIDE   # true bin centres (cells)
    print(f"RoI x = [{x1}, {x2}] px   true bin centres {np.round(true_c, 3)}  "
          f"RoIPool {np.round(roi_pool_avg(F, x1, x2), 3)}  RoIAlign {np.round(roi_align_avg(F, x1, x2), 3)}")

Output

RoI x = [22, 86] px   true bin centres [2.375 4.375]  RoIPool [2. 4.]  RoIAlign [2.375 4.375]
RoI x = [26, 90] px   true bin centres [2.625 4.625]  RoIPool [3. 5.]  RoIAlign [2.625 4.625]
RoI x = [30, 94] px   true bin centres [2.875 4.875]  RoIPool [3. 5.]  RoIAlign [2.875 4.875]

RoIAlign follows the box exactly, because bilinear interpolation reproduces a linear ramp. RoI pooling is off by 0.375, 0.375 and 0.125 cells, which is 6, 6 and 2 pixels at stride 16. Moving the RoI from the first row to the second by 4 pixels makes its output jump by a whole cell (16 pixels), and moving it again by 4 pixels changes nothing. A real feature map is not a ramp, but this staircase response is what RoI pooling does to the features.

How to read the paper’s evidence. Look first at Table 2 (c) and (d) of [16]. With a ResNet-50-C5 backbone, whose stride is 32, the rows RoIPool and RoIAlign give mask AP 23.6 and 30.9 (+7.3), and AP75 21.6 and 32.1 (+10.5); box AP75 rises from 26.9 to 36.4. The paper also tests RoIWarp, which interpolates bilinearly but still rounds the RoI, and finds it on a par with RoI pooling. So the gain comes from removing the rounding, not from the interpolation. When you read such an ablation, check that the compared rows share the backbone, the data split and the schedule (the table caption states training on trainval35k and testing on minival), and note that single numbers are reported; I found no repeated runs or variances in the text.

Common mistake. Forgetting the half-cell offset. If a cell value is treated as sitting at its corner while the RoI arithmetic assumes its center (or the reverse), every sample is read half a cell away, which is as large as the rounding error RoIAlign was designed to remove.

References

  1. Z. Zou, K. Chen, Z. Shi, Y. Guo and J. Ye, “Object detection in 20 years: A survey,” Proceedings of the IEEE, vol. 111, no. 3, 2023. doi
  2. L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu and M. Pietikäinen, “Deep learning for generic object detection: A survey,” International Journal of Computer Vision, 2020. doi
  3. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn and A. Zisserman, “The PASCAL Visual Object Classes (VOC) challenge,” International Journal of Computer Vision, vol. 88, 2010. doi
  4. T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick et al., “Microsoft COCO: Common objects in context,” in Proc. ECCV, 2014. doi
  5. P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proc. IEEE CVPR, 2001. doi
  6. N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE CVPR, 2005. doi
  7. P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, “Object detection with discriminatively trained part-based models,” IEEE TPAMI, 2010. doi
  8. R. Girshick, J. Donahue, T. Darrell and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proc. IEEE CVPR, 2014. arXiv
  9. R. Girshick, “Fast R-CNN,” in Proc. IEEE ICCV, 2015. arXiv
  10. S. Ren, K. He, R. Girshick and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” IEEE TPAMI, 2017 (conference version 2015). doi
  11. H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” in Proc. IEEE/CVF CVPR, 2019. arXiv
  12. Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye and D. Ren, “Distance-IoU loss: Faster and better learning for bounding box regression,” in Proc. AAAI, 2020. arXiv
  13. N. Bodla, B. Singh, R. Chellappa and L. S. Davis, “Soft-NMS — Improving object detection with one line of code,” in Proc. IEEE ICCV, 2017. arXiv
  14. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan and S. Belongie, “Feature pyramid networks for object detection,” in Proc. IEEE CVPR, 2017. doi
  15. T.-Y. Lin, P. Goyal, R. Girshick, K. He and P. Dollár, “Focal loss for dense object detection,” in Proc. IEEE ICCV, 2017. doi
  16. K. He, G. Gkioxari, P. Dollár and R. Girshick, “Mask R-CNN,” in Proc. IEEE ICCV, 2017. doi
  17. Z. Cai and N. Vasconcelos, “Cascade R-CNN: Delving into high quality object detection,” in Proc. IEEE/CVF CVPR, 2018. doi
  18. J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proc. IEEE CVPR, 2016. doi
  19. J. Redmon and A. Farhadi, “YOLO9000: Better, faster, stronger,” IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7263–7271, 2017. CVF · arXiv
  20. J. Redmon and A. Farhadi, “YOLOv3: An incremental improvement,” tech. report, arXiv:1804.02767, 2018. arXiv
  21. A. Bochkovskiy, C.-Y. Wang and H.-Y. M. Liao, “YOLOv4: Optimal speed and accuracy of object detection,” arXiv:2004.10934, 2020. arXiv
  22. C.-Y. Wang, A. Bochkovskiy and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” in Proc. IEEE/CVF CVPR, 2023. doi
  23. C.-Y. Wang, I-H. Yeh and H.-Y. M. Liao, “YOLOv9: Learning what you want to learn using programmable gradient information,” in Proc. ECCV, 2024. doi
  24. A. Wang, H. Chen, L. Liu, K. Chen, Z. Lin et al., “YOLOv10: Real-time end-to-end object detection,” in Proc. NeurIPS, 2024. arXiv
  25. Y. Tian, Q. Ye and D. Doermann, “YOLOv12: Attention-centric real-time object detectors,” arXiv:2502.12524, 2025. arXiv
  26. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu and A. C. Berg, “SSD: Single shot multibox detector,” in Proc. ECCV, 2016. arXiv
  27. H. Law and J. Deng, “CornerNet: Detecting objects as paired keypoints,” in Proc. ECCV, 2018. doi
  28. X. Zhou, D. Wang and P. Krähenbühl, “Objects as points,” arXiv:1904.07850, 2019. arXiv
  29. Z. Tian, C. Shen, H. Chen and T. He, “FCOS: Fully convolutional one-stage object detection,” in Proc. IEEE/CVF ICCV, 2019. arXiv
  30. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. ECCV, 2020. doi
  31. H. W. Kuhn, “The Hungarian method for the assignment problem,” Naval Research Logistics Quarterly, vol. 2, 1955. doi
  32. X. Zhu, W. Su, L. Lu, B. Li, X. Wang and J. Dai, “Deformable DETR: Deformable transformers for end-to-end object detection,” in Proc. ICLR, 2021. arXiv
  33. H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni and H.-Y. Shum, “DINO: DETR with improved denoising anchor boxes for end-to-end object detection,” arXiv:2203.03605, 2022. arXiv
  34. Y. Zhao, W. Lv, S. Xu, J. Wei, G. Wang et al., “DETRs beat YOLOs on real-time object detection,” in Proc. IEEE/CVF CVPR, 2024. doi
  35. M. Minderer, A. Gritsenko, A. Stone, M. Neumann, D. Weissenborn et al., “Simple open-vocabulary object detection with vision transformers,” in Proc. ECCV, 2022. arXiv
  36. S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang et al., “Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection,” in Proc. ECCV, 2024. doi
  37. T. Cheng, L. Song, Y. Ge, W. Liu, X. Wang et al., “YOLO-World: Real-time open-vocabulary object detection,” in Proc. IEEE/CVF CVPR, 2024. doi
  38. N. Carion, L. Gustafson, Y.-T. Hu, S. Debnath, R. Hu et al., “SAM 3: Segment anything with concepts,” arXiv:2511.16719, 2025. arXiv
  39. G. Cheng, X. Yuan, X. Yao, K. Yan, Q. Zeng et al., “Towards large-scale small object detection: Survey and benchmarks,” IEEE TPAMI, vol. 45, no. 11, 2023. arXiv
  40. COCO Consortium, “COCO detection evaluation,” official evaluation description (12 metrics, 101-point interpolation). link · official evaluator
  41. M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn and A. Zisserman, “The PASCAL Visual Object Classes challenge: A retrospective,” International Journal of Computer Vision, vol. 111, 2015. doi
  42. S. Shao, Z. Li, T. Zhang, C. Peng et al., “Objects365: A large-scale, high-quality dataset for object detection,” in Proc. IEEE/CVF ICCV, 2019. doi
  43. A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin et al., “The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale,” International Journal of Computer Vision, 2020. arXiv
  44. A. Gupta, P. Dollár and R. Girshick, “LVIS: A dataset for large vocabulary instance segmentation,” in Proc. IEEE/CVF CVPR, 2019. doi
  45. T. Shehzadi, K. A. Hashmi, M. Liwicki, D. Stricker and M. Z. Afzal, “Object detection with transformers: A review,” Sensors, vol. 25, no. 19, 2025. doi
  46. C. Zhu and L. Chen, “A survey on open-vocabulary detection and segmentation: Past, present, and future,” IEEE TPAMI, 2024. doi