當我們得知兩張圖之間的對應點、相機參數和座標轉換時,我們還可以透過 Triangulation 的到這些對應點的三維座標,與以前數學課程中學過的三角測量基本相同,只是放到了三維空間,並且多了一個投影函數。
如下圖,假設已知兩個藍色的點是對應的(使用前面幾章的特徵點匹配),我們就可以沿著兩個相機的中心連接這兩個點,這樣就可以得到兩條直線,理論上這兩條直線會在三維空間的一點相交,這個點就是我們要找的三維座標。
計算的方法是利用我們前面提過的投影公式:
已知兩組對應點 ,以及剩下的相機參數,我們可以求解 3D 點的座標 。現實中,相機的參數或是對應點的座標都有一些誤差,所以我們想像中的兩條直線不會完全相交,而我們求的解就是讓這兩條直線的距離最小的點。
OpenCV 提供了 cv2.triangulatePoints
的函數,可以很簡單的得到這個三維座標,以下是一個簡單的範例:
def triangulate_points(
points1, points2, intrinsic, camera_to_world1, camera_to_world2
):
"""
Args:
points1: 2D points in the first image (N, 2)
points2: 2D points in the second image (N, 2)
intrinsic: Intrinsic matrix of the camera (3, 3)
camera_to_world1: Camera-to-world matrix of the first image (4, 4)
camera_to_world2: Camera-to-world matrix of the second image (4, 4)
"""
assert len(points1) == len(points2)
# Convert camera-to-world to world-to-camera
world_to_camera1 = np.linalg.inv(camera_to_world1)
world_to_camera2 = np.linalg.inv(camera_to_world2)
# Compute projection matrix
P1 = intrinsic @ world_to_camera1[:3, :4]
P2 = intrinsic @ world_to_camera2[:3, :4]
points_3d = cv2.triangulatePoints(P1, P2, points1.T, points2.T)
points_3d = points_3d.T
# From homogeneous (N, 4) to (N, 3)
return points_3d[:, :3] / points_3d[:, 3:]
def create_point_cloud(points, colors, parent=None):
# Draw point cloud using the point xyz and colors
colors = colors.astype(np.float32) / 255.0
for i in range(len(points)):
point = scene.visuals.Sphere(radius=0.02, method="latitude", color=colors[i], parent=parent)
point.transform = scene.transforms.STTransform(translate=points[i])
def main():
.... # Previous example
camera_to_world_est = np.eye(4)
camera_to_world_est[:3, :3] = R
camera_to_world_est[:3, 3] = t.ravel()
create_frustum_with_image(rgb1, np.eye(4), color="blue")
create_frustum_with_image(rgb2, camera_to_world_est, color="blue")
points_3d = triangulate_points(
points1, points2, intrinsic, np.eye(4), camera_to_world_est
)
color = rgb1[points1[:, 1].astype(np.int32), points1[:, 0].astype(np.int32)]
create_point_cloud(points_3d, color, parent=view.scene)
就可以得到以下的結果: