September 15, 2003 ICRA2003, Taipei Lecture G: Multiple-View Lecture G: Multiple-View Reconstruction from Scene Knowledge Reconstruction from Scene Knowledge Yi Ma Yi Ma Perception & Decision Laboratory Decision & Control Group, CSL Image Formation & Processing Group, Beckman Electrical & Computer Engineering Dept., UIUC http://decision.csl.uiuc.edu/~yima Breakthroughs in 3D Reconstruction and Motion Analysis
45
Embed
Lecture G: Multiple-View Reconstruction from Scene Knowledge
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
September 15, 2003 ICRA2003, Taipei
Lecture G: Multiple-View Lecture G: Multiple-View Reconstruction from Scene KnowledgeReconstruction from Scene Knowledge
Reconstruction (from single view): Kanade’81, Fawcett’93,Rothwell’93, Zabrodsky’95’97, van Gool et.al.’96, Carlsson’98, Svedberg and Carlsson’99, Francois and Medioni’02, Huang, Yang, Hong, Ma’02,03
September 15, 2003 ICRA2003, Taipei
MUTIPLE-VIEW MULTIPLE-OBJECT ALIGNMENT• Scale alignment: adjacent objects in a single view• Scale alignment: same object in multiple views
SUMMARY: Problems and future work
ALGORITHMS & EXAMPLES• Building 3-D geometric models with symmetry • Symmetry extraction, detection, and matching • Camera calibration
SYMMETRY & MULTIPLE-VIEW GEOMETRY• Fundamental types of symmetry• Equivalent views• Symmetry based reconstruction
Multiple-View Reconstruction from Scene Knowledge
September 15, 2003 ICRA2003, Taipei
SYMMETRY & MUTIPLE-VIEW GEOMETRY
• Why does an image of a symmetric object give away its structure?
• Why does an image of a symmetric object give away its pose?
• What else can we get from an image of a symmetric object?
September 15, 2003 ICRA2003, Taipei
Equivalent views from rotational symmetry
90O
September 15, 2003 ICRA2003, Taipei
Equivalent views from reflectional symmetry
September 15, 2003 ICRA2003, Taipei
Equivalent views from translational symmetry
September 15, 2003 ICRA2003, Taipei
GEOMETRY FOR SINGLE IMAGES – Symmetric Structure
Definition. A set of 3-D features S is called a symmetric structureif there exists a nontrivial subgroup G of E(3) that acts on it such that for every g in G, the map
is an (isometric) automorphism of S. We say the structure S has agroup symmetry G.
September 15, 2003 ICRA2003, Taipei
GEOMETRY FOR SINGLE IMAGES – Multiple “Equivalent” Views
September 15, 2003 ICRA2003, Taipei
GEOMETRY FOR SINGLE IMAGES – Symmetric Rank Condition
Solving g0 from Lyapunov equations:
with g’i and gi known.
September 15, 2003 ICRA2003, Taipei
THREE TYPES OF SYMMETRY – Reflective Symmetry
Pr
September 15, 2003 ICRA2003, Taipei
THREE TYPES OF SYMMETRY – Rotational Symmetry
September 15, 2003 ICRA2003, Taipei
THREE TYPES OF SYMMETRY – Translatory Symmetry
September 15, 2003 ICRA2003, Taipei
SINGLE-VIEW GEOMETRY WITH SYMMETRY – Ambiguities
“(a+b)-parameter” means there are an a-parameter family of ambiguity in R0 of g0 and a b-parameter family of ambiguity in T0 of g0.
P
Pr
NPr
September 15, 2003 ICRA2003, Taipei
Symmetry-based reconstruction (reflection)
Reflectional symmetry
Virtual camera-camera
1 2
3 4(3)(4)
(2) (1)
September 15, 2003 ICRA2003, Taipei
Epipolar constraint
Homography
1 2
3 4(3)(4)
(2) (1)
Symmetry-based reconstruction
September 15, 2003 ICRA2003, Taipei
2 pairs of symmetric image points
Decompose or to obtain
Solve Lyapunov equation
to obtain and then .
Recover essential matrix or homography
Symmetry-based reconstruction (algorithm)
September 15, 2003 ICRA2003, Taipei
Symmetry-based reconstruction (reflection)
September 15, 2003 ICRA2003, Taipei
Symmetry-based reconstruction (rotation)
September 15, 2003 ICRA2003, Taipei
Symmetry-based reconstruction (translation)
September 15, 2003 ICRA2003, Taipei
ALIGNMENT OF MULTIPLE SYMMETRIC OBJECTS
?
September 15, 2003 ICRA2003, Taipei
Pick the image of a point on the intersection line
Correct scales within a single image
September 15, 2003 ICRA2003, Taipei
Correct scale within a single image
September 15, 2003 ICRA2003, Taipei
Correct scales across multiple images
September 15, 2003 ICRA2003, Taipei
Correct scales across multiple images
September 15, 2003 ICRA2003, Taipei
Correct scales across multiple images
September 15, 2003 ICRA2003, Taipei
Image alignment after scales corrected
September 15, 2003 ICRA2003, Taipei
ALGORITHM: Building 3-D geometric models
1. Specify symmetric objects and correspondence
September 15, 2003 ICRA2003, Taipei
Building 3-D geometric models
2. Recover camera poses and scene structure
September 15, 2003 ICRA2003, Taipei
Building 3-D geometric models
3. Obtain 3-D model and rendering with images
September 15, 2003 ICRA2003, Taipei
ALGORITHM: Symmetry detection and matching
Extract, detect, match symmetric objects acrossimages, and recover the camera poses.
5. Find the only one set of camera poses that are consistent with all symmetry objects
September 15, 2003 ICRA2003, Taipei
MATCHING OF SYMMETRY CELLS – Graph Representation
123
123
36 possible matchings
(1,2,1, g)Cell in image 1
Cell in image 2
# of possible matching
Camera transformation
The problem of finding the largest set of matching cells is equivalent to the problem of finding the maximal complete subgraphs (cliques) in the matching graph.
September 15, 2003 ICRA2003, Taipei
Camera poses and 3-D recovery Side view Top view Generic view
Length ratio Reconstruction Ground truthWhiteboard 1.506 1.51Table 1.003 1.00
September 15, 2003 ICRA2003, Taipei
Multiple-view matching and recovery (Ambiguities)
September 15, 2003 ICRA2003, Taipei
Multiple-view matching and recovery (Ambiguities)
September 15, 2003 ICRA2003, Taipei
ALGORITHM: Calibration from symmetry
(vanishing point)
Calibrated homography
Uncalibrated homography
September 15, 2003 ICRA2003, Taipei
ALGORITHM: Calibration from symmetry
Calibration with a rig is also simplified: we only need to know that there are sufficient symmetries, not necessarily the 3-D coordinates of points on the rig.