7/30/2019 3D reconstruction Slides
1/12
3D Reconstruction
Tutorial
By
(Waqar S. Qureshi)
Vision and Graphics Lab, Asian Institute ofTechnology, Thailand
7/30/2019 3D reconstruction Slides
2/12
What is 3D-Reconstruction?
In computer vision, 3D reconstruction is amethod to find the shape of an object or theappearance of a scene in an image[1].
This requires to calculate the depth estimatesof the 2D-points of an image of a 3D scene orobject.
Today, we will discuss how to get 3D-pointsestimates using monocular camera
[1] http://en.wikipedia.org/wiki/3D_reconstruction
7/30/2019 3D reconstruction Slides
3/12
3D-Reconstruction from image sequence
What do we need
Appearance of a 3D-scene, shape of theobject of interest
3D-points or depth estimates of a scene orobject
What do we have
Sequence of Images from a monocularcamera
How to solve it?
Use Multi-view geometry and linear algebra
7/30/2019 3D reconstruction Slides
4/12
7/30/2019 3D reconstruction Slides
5/12
3D-Reconstruction
Types:
Spacial 3D-reconstruction
2-D feature correspondences in spacial domain
are extracted and then 3D depths areestimated
Dense 2-D feature correspondences in spacialdomain are extracted and then 3D depths areestimated
7/30/2019 3D reconstruction Slides
6/12
3D-Reconstruction
Overview of the method taken from Atima M.Sc thesis presentation AIT
7/30/2019 3D reconstruction Slides
7/12
SURF Correspondences
[1]- Find SURF correspondences between two images as follows:
Find the Key-point and descriptor in the first image.
Find the Key-point and descriptor in the second image.
Find the Matching Key-points correspondence from image-1 to image-2
using K-nearest neighbor.
Find the Matching Key-points correspondence from image-2 to image-1using K-nearest neighbor.
Remove all the non-symmetric correspondences in both the images.
Remove all the correspondences which are not consistent with theepipolar geometry (Fundamental Matrix).
Find the fundamental matrix with only the left-over matches.
7/30/2019 3D reconstruction Slides
8/12
SURF Correspondences
7/30/2019 3D reconstruction Slides
9/12
3D-Reconstruction
[1] - Find the pair of frame having the best GRIC ratio and use themto initialize reconstruction. Use only SURF correspondences to findthe fundamental matrix and the homography matrix for calculating theGRIC.
[2] - Recover the motion for the pair of frames. We call it aInitialization/Recover Motion
Get correspondence between the frames
Get Camera P1 and P2 using essential matrix method by Hertley (2004).
Use triangulation to find the 3D-points using the equation X . P = x
Check if enough points lie in front of the cameras.(only 10-15 outliers)
Add the pair of recovered cameras to the vector of cameras.
7/30/2019 3D reconstruction Slides
10/12
Triangulation
[3] - Estimate the 3D points using cameras recovered in step 2. This isknown as Triangulation
Find the dense correspondence between the frames to be triangulated
Check if initialization=True
if initialization=True then find all the new 2D-point correspondences in the set of
correspondences which are not already recovered.
Get camera matrices P1 and P2 from camera matrices set for frames.
For new 2D-points correspondences find the 3D points (triangulate).
Find the re-projection error for the 3D points recovered in previous step and remove
outliers Check if the points lie in-front of the cameras.
Add the 3D-points to the vector of 3D points.
7/30/2019 3D reconstruction Slides
11/12
3D-Reconstruction
[4] - Set the initialization = TRUE.
[5] - Resection the next i-th frame
Check if Initialization=True if not return
For the number of times we need to mergeframes do...
7/30/2019 3D reconstruction Slides
12/12
Re-sectioning
Get the point-correspondences between the i-th frame to be re-sectionedand the j-th frame recovered.
Find the new 2Dto3D correspondences by matching the pointcorrespondences from step ii to 2D-projections of the i-th frame in the 3D-points vectors.
If the 2Dto3D correspondences are not greater then threshold do not dore-sectioning for this frame.
Get all the 2Dto3D pairs and find Camera matrix P using Ransack.
Remove the outliers and find the camera matrix P with all the inlier usingDLT.
Add camera to the vector of cameras.
[6] - Save all the 3D-Point cloud with their corresponding 2D projections
and frame Ids for visualization.