完整搜尋頁面 →

線性三角測量(DLT)

選讀・DLCV 11・三維重建:從多視角幾何到 NeRF 與 Gaussian Splatting 的補充資料(第 2/7 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 7

  1. 正規化八點演算法
  2. 線性三角測量(DLT)
  3. 程式碼中的光束法平差
  4. 平面誘導的單應性
  5. 多解析度雜湊編碼
  6. 為什麼 NeuS 的權重會在表面上達到峰值
  7. 投影 3D 高斯:局部仿射近似

論文導讀 · 3

  1. NeRF(Mildenhall 等人,2020)
  2. 3D Gaussian Splatting(Kerbl 等人,2023)
  3. 「現代觀點」綜述導讀

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

這則筆記把正文用的線性三角測量完整寫出來,說明為什麼它的答案不是最佳解,並量測基線如何控制它的誤差。

方程式。 把未知點寫成齊次形式 X~=(X,Y,Z,1)⊤\tilde{\mathbf{X}} = (X, Y, Z, 1)^\top,令 pik⊤\mathbf{p}_i^{k\top} 為相機 ii 的投影矩陣 PiP_i 的第 kk 列。針孔模型說 λi(ui,vi,1)⊤=PiX~\lambda_i (u_i, v_i, 1)^\top = P_i \tilde{\mathbf{X}},所以深度是 λi=pi3⊤X~\lambda_i = \mathbf{p}_i^{3\top} \tilde{\mathbf{X}}。把它代入前兩列,就消去未知的深度,每張影像留下兩個線性方程式:

AX~=0,A=[u1p13⊤−p11⊤v1p13⊤−p12⊤u2p23⊤−p21⊤v2p23⊤−p22⊤],A \tilde{\mathbf{X}} = \mathbf{0}, \qquad A = \begin{bmatrix} u_1 \mathbf{p}_1^{3\top} - \mathbf{p}_1^{1\top} \\ v_1 \mathbf{p}_1^{3\top} - \mathbf{p}_1^{2\top} \\ u_2 \mathbf{p}_2^{3\top} - \mathbf{p}_2^{1\top} \\ v_2 \mathbf{p}_2^{3\top} - \mathbf{p}_2^{2\top} \end{bmatrix},

其中 (ui,vi)(u_i, v_i) 是影像 ii 中觀測到的像素,AA 是 4×44 \times 4 的矩陣。這就是直接線性轉換(direct linear transformation, DLT)法 [1]。有雜訊時,AA 沒有精確的零向量。使 ∥AX~∥\lVert A \tilde{\mathbf{X}} \rVert 最小的單位向量,是 AA 最小奇異值所對應的右奇異向量;把它除以最後一個元素,就得到歐氏空間中的點。每多一個視角就多兩列加進 AA,其他都不變。

為什麼它不是最佳的。 被最小化的量 ∥AX~∥\lVert A \tilde{\mathbf{X}} \rVert 是代數誤差:它在像素上沒有直接的意義,而且影像座標重新縮放時它會跟著改變。在高斯像素雜訊下,合理的目標是重投影誤差,也就是 X\mathbf{X} 的投影與觀測點之間像素距離的平方和。Hartley 與 Zisserman [1] 描述了直接最小化它的方法。實務上,DLT 的答案是起點,之後再由光束法平差一起精修所有的點與相機。

邊界情況。 (i) 基線短或點很遠時,兩條射線幾乎平行,AA 接近秩 2,深度變得非常不確定。(ii) 無窮遠的點,其最後一個齊次座標為 0,所以最後的除法會失敗;這類點必須保持齊次形式,或者移除。(iii) 錯誤的配對可能產生位於相機後方的點,所以除以最後一個元素之後,要檢查深度 pi3⊤X~\mathbf{p}_i^{3\top} \tilde{\mathbf{X}} 在每個視角中都是正的(正文的手性檢查)。

測試。 這段程式碼對位於相機 1 前方 6 單位的一個點做三角測量,由兩台帶 0.5 像素雜訊的相機觀測,並使用三種基線。它把深度誤差的散布和正文的一階公式 δZ≈Z2δd/(fB)\delta Z \approx Z^2 \delta d / (fB) 比較;視差誤差是兩個獨立 0.5 像素誤差的差,標準差為 0.520.5\sqrt{2} 像素。

程式碼 · Python · 31 行
import numpy as np
import cv2

rng = np.random.default_rng(0)
K = np.array([[800.0, 0, 320], [0, 800.0, 240], [0, 0, 1]])
X_true = np.array([0.3, -0.2, 6.0])                     # one point, 6 units in front of camera 1

def triangulate_dlt(P1, P2, x1, x2):
    """Linear triangulation: four equations A X = 0 in the homogeneous point X (4-vector)."""
    A = np.stack([x1[0] * P1[2] - P1[0], x1[1] * P1[2] - P1[1],
                  x2[0] * P2[2] - P2[0], x2[1] * P2[2] - P2[1]])
    Xh = np.linalg.svd(A)[2][-1]                         # right singular vector, smallest singular value
    return Xh[:3] / Xh[3]                                # homogeneous -> Euclidean

def proj(P, X):
    x = P @ np.append(X, 1.0)
    return x[:2] / x[2]

P1 = K @ np.hstack([np.eye(3), np.zeros((3, 1))])
for B in [2.0, 0.5, 0.1]:                                # camera 2 sits B to the right, same orientation
    P2 = K @ np.hstack([np.eye(3), [[-B], [0.0], [0.0]]])
    dz = []
    for _ in range(2000):
        x1 = proj(P1, X_true) + rng.normal(0, 0.5, 2)      # 0.5 px noise in each image
        x2 = proj(P2, X_true) + rng.normal(0, 0.5, 2)
        dz.append(triangulate_dlt(P1, P2, x1, x2)[2] - X_true[2])
    predicted = X_true[2] ** 2 * (0.5 * np.sqrt(2)) / (800 * B)   # Z^2 * std(disparity error) / (f B)
    print(f"baseline {B:3.1f}: ray angle {np.degrees(np.arctan2(B, 6)):5.2f} deg | "
          f"depth error std {np.std(dz):.4f} (first-order prediction {predicted:.4f})")
Xcv = cv2.triangulatePoints(P1, P2, x1.reshape(2, 1), x2.reshape(2, 1)).ravel()
print(f"difference from cv2.triangulatePoints: {np.abs(Xcv[:3] / Xcv[3] - triangulate_dlt(P1, P2, x1, x2)).max():.1e}")

輸出

baseline 2.0: ray angle 18.43 deg | depth error std 0.0158 (first-order prediction 0.0159)
baseline 0.5: ray angle  4.76 deg | depth error std 0.0617 (first-order prediction 0.0636)
baseline 0.1: ray angle  0.95 deg | depth error std 0.3166 (first-order prediction 0.3182)
difference from cv2.triangulatePoints: 2.0e-13

量到的散布與一階預測吻合。把基線從 2 縮到 0.1 單位,使兩條射線的夾角小於一度,深度誤差就乘以 20。最後一行顯示 OpenCV 的 cv2.triangulatePoints 回傳的是同一個點,差異只有捨入誤差。

參考文獻

  1. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed., Cambridge University Press, 2004. doi
  2. J. L. Schönberger and J.-M. Frahm, “Structure-from-motion revisited,” in Proc. IEEE CVPR, 2016. CVF
  3. J. L. Schönberger, E. Zheng, J.-M. Frahm and M. Pollefeys, “Pixelwise view selection for unstructured multi-view stereo,” in Proc. ECCV, 2016. doi
  4. J. L. Schönberger et al., “COLMAP,” official documentation (general-purpose structure-from-motion and multi-view stereo pipeline). link
  5. B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,” in Proc. ECCV, 2020; also Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2022. doi · CACM doi · arXiv
  6. B. Kerbl, G. Kopanas, T. Leimkühler and G. Drettakis, “3D Gaussian Splatting for real-time radiance field rendering,” ACM Transactions on Graphics, vol. 42, no. 4, 2023. doi · arXiv
  7. H. C. Longuet-Higgins, “A computer algorithm for reconstructing a scene from two projections,” Nature, vol. 293, pp. 133–135, 1981. doi
  8. R. I. Hartley, “In defense of the eight-point algorithm,” IEEE TPAMI, vol. 19, no. 6, pp. 580–593, 1997. doi
  9. D. Nistér, “An efficient solution to the five-point relative pose problem,” IEEE TPAMI, vol. 26, no. 6, pp. 756–770, 2004. doi
  10. M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. doi
  11. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. doi
  12. N. Snavely, S. M. Seitz and R. Szeliski, “Photo tourism: Exploring photo collections in 3D,” ACM Transactions on Graphics, vol. 25, no. 3, pp. 835–846, 2006. doi
  13. B. Triggs, P. F. McLauchlan, R. I. Hartley and A. W. Fitzgibbon, “Bundle adjustment — A modern synthesis,” in Vision Algorithms: Theory and Practice, LNCS 1883, pp. 298–372, Springer, 2000. doi
  14. R. T. Collins, “A space-sweep approach to true multi-image matching,” in Proc. IEEE CVPR, pp. 358–363, 1996. doi
  15. B. Curless and M. Levoy, “A volumetric method for building complex models from range images,” in Proc. SIGGRAPH, pp. 303–312, 1996. doi
  16. Y. Yao, Z. Luo, S. Li, T. Fang and L. Quan, “MVSNet: Depth inference for unstructured multi-view stereo,” in Proc. ECCV, 2018. CVF · doi
  17. N. Max, “Optical models for direct volume rendering,” IEEE Transactions on Visualization and Computer Graphics, vol. 1, no. 2, pp. 99–108, 1995. doi
  18. M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” in Proc. NeurIPS, 2020. proceedings
  19. T. Müller, A. Evans, C. Schied and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM Transactions on Graphics, vol. 41, no. 4, 2022. doi · arXiv
  20. J. J. Park, P. Florence, J. Straub, R. Newcombe and S. Lovegrove, “DeepSDF: Learning continuous signed distance functions for shape representation,” in Proc. IEEE/CVF CVPR, 2019. CVF
  21. L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin and A. Geiger, “Occupancy networks: Learning 3D reconstruction in function space,” in Proc. IEEE/CVF CVPR, 2019. CVF
  22. P. Wang, L. Liu, Y. Liu, C. Theobalt, T. Komura and W. Wang, “NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction,” in Proc. NeurIPS, 2021. proceedings · arXiv
  23. M. Zwicker, H. Pfister, J. van Baar and M. Gross, “EWA volume splatting,” in Proc. IEEE Visualization, 2001. doi
  24. W. E. Lorensen and H. E. Cline, “Marching cubes: A high resolution 3D surface construction algorithm,” in Proc. SIGGRAPH, pp. 163–169, 1987. doi
  25. M. Kazhdan, M. Bolitho and H. Hoppe, “Poisson surface reconstruction,” in Proc. Eurographics Symposium on Geometry Processing, 2006. doi
  26. M. Kazhdan and H. Hoppe, “Screened Poisson surface reconstruction,” ACM Transactions on Graphics, vol. 32, no. 3, 2013. doi
  27. A. Guédon and V. Lepetit, “SuGaR: Surface-aligned Gaussian splatting for efficient 3D mesh reconstruction and high-quality mesh rendering,” in Proc. IEEE/CVF CVPR, 2024. CVF · arXiv
  28. B. Huang, Z. Yu, A. Chen, A. Geiger and S. Gao, “2D Gaussian splatting for geometrically accurate radiance fields,” in Proc. SIGGRAPH (Conference Papers), 2024. doi · arXiv
  29. S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii and J. Revaud, “DUSt3R: Geometric 3D vision made easy,” in Proc. IEEE/CVF CVPR, 2024. CVF · arXiv
  30. V. Leroy, Y. Cabon and J. Revaud, “Grounding image matching in 3D with MASt3R,” in Proc. ECCV, 2024. doi · arXiv
  31. J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht and D. Novotny, “VGGT: Visual geometry grounded transformer,” in Proc. IEEE/CVF CVPR, 2025. CVF · arXiv
  32. A. Pumarola, E. Corona, G. Pons-Moll and F. Moreno-Noguer, “D-NeRF: Neural radiance fields for dynamic scenes,” in Proc. IEEE/CVF CVPR, 2021. CVF
  33. G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian and X. Wang, “4D Gaussian splatting for real-time dynamic scene rendering,” in Proc. IEEE/CVF CVPR, 2024. CVF · arXiv
  34. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  35. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proc. IEEE/CVF CVPR, 2018. CVF
  36. H. Fan, H. Su and L. J. Guibas, “A point set generation network for 3D object reconstruction from a single image,” in Proc. IEEE CVPR, 2017. CVF
  37. R. Jensen, A. Dahl, G. Vogiatzis, E. Tola and H. Aanæs, “Large scale multi-view stereopsis evaluation,” in Proc. IEEE CVPR, pp. 406–413, 2014. doi
  38. A. Knapitsch, J. Park, Q.-Y. Zhou and V. Koltun, “Tanks and Temples: Benchmarking large-scale scene reconstruction,” ACM Transactions on Graphics, vol. 36, no. 4, 2017. doi · benchmark
  39. J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan and P. Hedman, “Mip-NeRF 360: Unbounded anti-aliased neural radiance fields,” in Proc. IEEE/CVF CVPR, 2022. CVF · arXiv
  40. D. Verbin, P. Hedman, B. Mildenhall, T. Zickler, J. T. Barron and P. P. Srinivasan, “Ref-NeRF: Structured view-dependent appearance for neural radiance fields,” in Proc. IEEE/CVF CVPR, 2022. CVF
  41. C.-H. Lin, W.-C. Ma, A. Torralba and S. Lucey, “BARF: Bundle-adjusting neural radiance fields,” in Proc. IEEE/CVF ICCV, 2021. CVF
  42. S. M. Seitz, B. Curless, J. Diebel, D. Scharstein and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proc. IEEE CVPR, vol. 1, pp. 519–528, 2006. doi
  43. Y. Furukawa and C. Hernández, “Multi-view stereo: A tutorial,” Foundations and Trends in Computer Graphics and Vision, vol. 9, no. 1–2, pp. 1–148, 2015. doi
  44. A. Tewari, J. Thies, B. Mildenhall, P. Srinivasan, E. Tretschk, W. Yifan, C. Lassner, V. Sitzmann et al., “Advances in neural rendering,” Computer Graphics Forum, vol. 41, no. 2, pp. 703–735, 2022. doi · arXiv
  45. G. Chen and W. Wang, “A survey on 3D Gaussian splatting,” ACM Computing Surveys, vol. 58, no. 12, 2026. doi · arXiv
  46. scikit-image contributors, “skimage.measure.marching_cubes,” scikit-image documentation (API reference). docs
  47. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. ICLR, 2021. arXiv
  48. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser and I. Polosukhin, “Attention is all you need,” in Proc. NeurIPS, 2017. proceedings
  49. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. ICLR, 2015. arXiv