DLCV 03B · Is a small yaw error harmless?
AdvancedDLCV 03B · 3D Object Detection: LiDAR, Cameras and Bird's-Eye ViewImage lab
Description
Turn a predicted box a few degrees away from the ground truth in the bird’s-eye view and watch the exact rotated IoU fall while the centre distance stays at zero.
Try this: At the default setting (bus or truck, 12 × 2.5 m, yaw error 10°, centres identical) the rotated IoU is 0.657 although the centre distance is 0 m; the enclosing rectangles, which ignore the yaw, give 0.539. Lower the yaw error to 5° and the IoU is still only 0.808. Switch to the car (4.5 × 1.8 m) and the same 10° costs less, 0.796, then to the pedestrian (0.7 × 0.7 m): 0.863 at 10°, and never below 0.707 at any yaw error. The last row of the table gives the yaw error at which each shape falls below IoU 0.7: 8.5° for the bus and 16.55° for the car. Which class would a matching rule that uses an IoU threshold of 0.7 punish most for a heading mistake, and what does a rule that only looks at centre distance say about all three?
Explanation
Answer the prediction first, or open the explanation when you are ready.
Show the explanation
About 0.65. The 12 m box is so long that turning it by 10° swings each end about 1 m sideways, while the box is only 2.5 m wide, so a good part of the overlap is lost: the exact IoU is 0.657 (0.808 at 5°, 0.423 at 20°), although the centre distance is 0 m. This bus-sized box falls below IoU 0.7 at a yaw error of 8.5°, a car-sized one at 16.55°, and a square pedestrian box never does.
Lab · DLCV 03B · Is a small yaw error harmless?
Loading image…
Try this: At the default setting (bus or truck, 12 × 2.5 m, yaw error 10°, centres identical) the rotated IoU is 0.657 although the centre distance is 0 m; the enclosing rectangles, which ignore the yaw, give 0.539. Lower the yaw error to 5° and the IoU is still only 0.808. Switch to the car (4.5 × 1.8 m) and the same 10° costs less, 0.796, then to the pedestrian (0.7 × 0.7 m): 0.863 at 10°, and never below 0.707 at any yaw error. The last row of the table gives the yaw error at which each shape falls below IoU 0.7: 8.5° for the bus and 16.55° for the car. Which class would a matching rule that uses an IoU threshold of 0.7 punish most for a heading mistake, and what does a rule that only looks at centre distance say about all three?
The ground-truth box is axis-aligned at the origin with its length along x. The prediction has exactly the right size and is turned by the yaw error about its own centre, then moved along the ground truth’s length by the centre offset. Sizes are illustrative, the ones used in the chapter’s Figures 4 and 6 (pedestrian 0.7 × 0.7 m, car 4.5 × 1.8 m, bus or truck 12 × 2.5 m), not measurements of any dataset. Rotated IoU is exact: the overlap polygon comes from Sutherland–Hodgman clipping and its area from the shoelace formula, and IoU = overlap / (area of the ground truth + area of the prediction − overlap); the code is the chapter’s Python snippet ported to TypeScript and tested against it. The axis-aligned IoU takes the smallest axis-aligned rectangle around each box and computes the ordinary IoU of those two rectangles; it ignores the real shape, so it is lower than the rotated IoU for small yaw errors (the enclosing rectangle of a turned box is larger than the box) and higher for large ones. The centre distance is the Euclidean distance between the two centres on the ground plane. IoU cannot see a 180° flip of the heading, so the yaw error is limited to 0° to 90°. Only the bird’s-eye view is compared here (no height). The nuScenes benchmark matches a detection to an object by the 2D centre distance on the ground plane, to decouple detection from object size and orientation, and scores heading in a separate error term; KITTI’s car threshold of 0.7 is a 3D box overlap.
Method and image sources
- I. E. Sutherland and G. W. Hodgman, Reentrant Polygon Clipping, Communications of the ACM, 17(1), 32-42, 1974 (the polygon clipping used for the exact overlap)
- H. Caesar et al., nuScenes: A Multimodal Dataset for Autonomous Driving, CVPR, 2020 (matching by 2D centre distance on the ground plane instead of IoU)
- nuScenes devkit, nuScenes detection task (official evaluation documentation: centre-distance matching, orientation error)
- A. Geiger, P. Lenz, C. Stiller and R. Urtasun, The KITTI Vision Benchmark Suite: 3D object detection evaluation (3D box overlap of 70% for cars)