Computing Epipolar Geometry From Dynamic Silhouettes Synchronization and Calibration of a Camera Network for 3D Event Reconstruction from Live Video Sudipta N. Sinha Marc Pollefeys Department of Computer Science, University of North Carolina at Chapel Hill RANSAC-based Algorithm If the epipoles are known, then 3 pair of matching lines through the epipoles are required to compute the epipolar homography. RANSAC is used to randomly explore the 4D space of epipole hypothesis as well as robustly deal with incorrect silhouettes. Results : Pair-wise Epipolar Geometry on datasets. Goal : To Calibrate a Network of Cameras from video streams. Metric Camera Calibration from Fundamental (F) Matrices. Motivation: No Calibration Object (LED, grid) required. Handles wide baselines, lack of sufficient scene overlap, lack of enough point features and arbitrary camera configuration. Frontier Points and constraints provided by Epipolar Tangents. Compute Convex Hull + its Dual Hypothesis Step Verification Step Incremental Projective Reconstruction Given, F 12 , F 23 , F 13 compute consistent projective cameras P 1 , P 2 , P 3 Work in progress: Silhouet te Extracti on Compute Pair wise Epipolar Geometry using Silhouettes Metric Calibration Of Camera Network From Fundamental Matrices F 12 . . F jk P 1 P 2 . . P n Multiple Video Streams Funding: NSF Career IIS 0237533 & DARPA/DOI NBCH 1030015 Recovering Synchronization (Assume fixed known frame-rate) Add a random guess for temporal offset to the hypothesis. First, use slow-moving silhouettes for coarse sequence alignment and then use fast moving ones for recovering finer synchronization + epipolar geometry. Method of Induction used to add new cameras to existing network. The re-projected Visual Hull in one of the views Temporal Interpolation of Silhouettes MIT Dataset (4 views), 4 minutes of 30 fps video Synthetic Data (MPI Saarbrucken) (25 views, 200 frames) Presented at “Mathematical Methods in Computer Vision Workshop, Banff, Canada, Oct 2006 Name Camera s Frames Good F’s Projective Bundle Adj. MIT 4 7000 5 / 6 1580 pts 0.26 pixels INRIA 8 200 20 / 28 3243 pts 0.25 pixels KUNGFU 25 200 268 / 300 19632 pts 0.11 pixels GATECH 9 200 25 / 36 2503 pts 0.55 pixels CMU 8 900 22 / 28 5764 pts 0.47 pixels MPI 8 370 26 / 28 4975 pts 0.28 pixels SriKuma r 5 1000 9 / 10 1686 pts 0.24 pixels Incremental Projective Reconstruction + Projective Bundle Adjustment Self- Calibrat ion Euclidean Bundle Adjustmen t F 12 . . F jk P 1 P 2 . . P n 1. Projector Camera Calibration 2. Calibrating Hybrid Camera Networks Color, Range scanner, Depth sensor and Infra-red cameras. 3. Modify algorithm to work with fuzzy silhouettes to deal with gross silhouette extraction errors. MIT INRI A GATECH MPI CMU KUNGFU