One-shot Entire Shape Scanning by Utilizing Multiple Projector-Camera Constraints of Grid Patterns Nozomu Kasuya Kagoshima University Kagoshima, Japan [email protected],jp Ryusuke Sagawa AIST Tsukuba, Japan [email protected]Ryo Furukawa HIroshima City University Hiroshima, Japan [email protected]Hiroshi Kawasaki Kagoshima University Kagoshima, Japan [email protected]Abstract This paper proposes a method to reconstruct the entire shape of moving objects by using multiple cameras and projectors. The projectors simultaneously cast static grid patterns of wave lines. Each of the projected patterns is a single-colored pattern of either red, green, or blue. Those patterns can be decomposed stably, compared to multi- colored patterns. For the 3D reconstruction algorithm, one- shot reconstruction with wave grid pattern is extended for entire-shape acquisition, so that the correspondences be- tween the adjacent devices can be used as additional con- straints to reduce shape errors. Finally, multiple shapes obtained from the different views are merged into a single polygon mesh model using estimated normal information for each vertex. 1. Introduction In recent years, as practical and reliable active 3D scan- ning devices develop rapidly, strong demands on captur- ing motions of humans or animals as a series of entire 3D shapes is emerging [7]. Such 3D data can be applied to ges- ture recognition, markerless motion capture, digital fashion, and analysis of interaction between multiple humans or an- imals. An important technical issue for the entire-shape scan of a moving object is that a general 3D sensor can only cap- ture surfaces visible from one direction during one measure- ment; thus, acquiring an entire shape of an object is difficult. One approach that has been widely used is aligning a large number of cameras around the object, and reconstruct the 3D shape with shape-from-silhouette [3, 13] . These pas- sive measurement systems have achieved major successes; however, setting up those systems is complicated since they need a large number of cameras with broad networks and PCs, and the calculation time is huge. Active entire-shape measurement systems could be al- ternatives to overcome the above problems of the passive methods. However, since active methods use light sources, interferences between different illuminations become a ma- jor problem. Several methods that have been proposed un- til now can be largely separated into two types; one is to use different colors ( i.e., wave-lengths), the other is high- frequency switching between multiple light sources with synchronized camera capturing. Since the timing of captur- ing for multiple light sources are different, temporal inter- polation is needed for the integration of the shapes [20, 2], which may limit the extent of the applicability, the latter method is not suitable to scan fast-moving objects. There- fore, we take the former option in our method; however, the former method also has problems. If each of the light sources project a light pattern with multiple colors, it is difficult to decompose the light pat- terns from the captured image in which multiple patterns are overlapped on a same surface; thus, devices that just use single-colored pattern is suitable for the system, such as Kinect. However, just aligning multiple devices with such a property also has problems, such as inconsistency between shapes measured by difference devices that come from cal- ibration errors of intrinsic or extrinsic parameters. An approach to deal with this problem is using corre- spondences between multiple patterns projected from mul- tiple light projectors. As far as we know, there has been no method that performs this method with single-colored pat- terns. The method proposed by Furukawa et al. is related to 299 299
8
Embed
One-Shot Entire Shape Scanning by Utilizing Multiple ... · tor. If we use multiple cameras and projectors, additional constraints on the correspondence between two cameras, or two
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
This paper proposes a method to reconstruct the entireshape of moving objects by using multiple cameras andprojectors. The projectors simultaneously cast static gridpatterns of wave lines. Each of the projected patterns is asingle-colored pattern of either red, green, or blue. Thosepatterns can be decomposed stably, compared to multi-colored patterns. For the 3D reconstruction algorithm, one-shot reconstruction with wave grid pattern is extended forentire-shape acquisition, so that the correspondences be-tween the adjacent devices can be used as additional con-straints to reduce shape errors. Finally, multiple shapesobtained from the different views are merged into a singlepolygon mesh model using estimated normal informationfor each vertex.
1. Introduction
In recent years, as practical and reliable active 3D scan-
ning devices develop rapidly, strong demands on captur-
ing motions of humans or animals as a series of entire 3D
shapes is emerging [7]. Such 3D data can be applied to ges-
ture recognition, markerless motion capture, digital fashion,
and analysis of interaction between multiple humans or an-
imals.
An important technical issue for the entire-shape scan of
a moving object is that a general 3D sensor can only cap-
ture surfaces visible from one direction during one measure-
ment; thus, acquiring an entire shape of an object is difficult.
One approach that has been widely used is aligning a large
number of cameras around the object, and reconstruct the
3D shape with shape-from-silhouette [3, 13] . These pas-
sive measurement systems have achieved major successes;
however, setting up those systems is complicated since they
need a large number of cameras with broad networks and
PCs, and the calculation time is huge.
Active entire-shape measurement systems could be al-
ternatives to overcome the above problems of the passive
methods. However, since active methods use light sources,
interferences between different illuminations become a ma-
jor problem. Several methods that have been proposed un-
til now can be largely separated into two types; one is to
use different colors (i.e., wave-lengths), the other is high-
frequency switching between multiple light sources with
synchronized camera capturing. Since the timing of captur-
ing for multiple light sources are different, temporal inter-
polation is needed for the integration of the shapes [20, 2],
which may limit the extent of the applicability, the latter
method is not suitable to scan fast-moving objects. There-
fore, we take the former option in our method; however, the
former method also has problems.
If each of the light sources project a light pattern with
multiple colors, it is difficult to decompose the light pat-
terns from the captured image in which multiple patterns
are overlapped on a same surface; thus, devices that just
use single-colored pattern is suitable for the system, such as
Kinect. However, just aligning multiple devices with such a
property also has problems, such as inconsistency between
shapes measured by difference devices that come from cal-
ibration errors of intrinsic or extrinsic parameters.
An approach to deal with this problem is using corre-
spondences between multiple patterns projected from mul-
tiple light projectors. As far as we know, there has been no
method that performs this method with single-colored pat-
terns. The method proposed by Furukawa et al. is related to
2013 IEEE International Conference on Computer Vision Workshops
showed several gaps between shapes. In the future, we plan
to extend the proposed method to achieve real-time process-
ing using GPU and reconstruct a single surface directly.
AcknowledgmentThis work was supported in part by NEXT program
No.LR030 in Japan.
References[1] Artec. United States Patent Application 2009005924, 2007j.
2[2] Q. D. Chenglei Wu, Yebin Liu and B. Wilburn. Fusing multi-
view and photometric stereo for 3D reconstruction under un-calibrated illumination. IEEE Transactions on Visualizationand Computer Graphics, 17(8):1082–1095, 2011. 1, 2
[3] G. K. M. Cheung, T. Kanade, J.-Y. Bouguet, and M. Holler.A real time system for robust 3D voxel reconstruction of hu-man motions. In CVPR’00, pages 2714–2720, 2000. 1, 2
[4] P. Felzenszwalb and D. Huttenlocher. Efficient belief propa-gation for early vision. IJCV, 70:41–54, 2006. 4
[5] R. Furukawa, R. Sagawa, H. Kawasaki, K. Sakashita,Y. Yagi, and N. Asada. One-shot entire shape acquisi-tion method using multiple projectors and cameras. In 4thPacific-Rim Symposium on Image and Video Technology,pages 107–114. IEEE Computer Society, 2010. 2
[6] R. Furukawa, R. Sagawa, H. Kawasaki, K. Sakashita,Y. Yagi, and N. Asada. Entire shape acquisition techniqueusing multiple projectors and cameras with parallel pat-tern projection. IPSJ Transactions on Computer Vision andApplications, 4:40–52, Mar. 2012. 3
[7] J. Gall, C. Stoll, E. de Aguiar, C. Theobalt, B. Rosenhahn,and H.-P. Seidel. Motion capture using joint skeleton track-ing and surface estimation. In 2009 IEEE Conference onComputer Vision and Pattern Recognition : CVPR 2009,pages 1746–1753, Miami, USA, 2009. IEEE. 1
[8] N. Kasuya, R. Sagawa, R. Furukawa, and H. Kawasaki.Robust and accurate one-shot 3D reconstruction by 2C1Psystem with wave grid pattern. In Proc. InternationalConference on 3D Vision, Seattle, USA, June. 2013. 4, 6
[9] H. Kawasaki, R. Furukawa, , R. Sagawa, and Y. Yagi. Dy-namic scene shape reconstruction using a single structuredlight pattern. In CVPR, pages 1–8, June 23-28 2008. 2
[10] M. Kazhdan, M. Bolitho, and H. Hoppe. Poisson surfacereconstruction. In Proceedings of the fourth Eurographicssymposium on Geometry processing, SGP ’06, pages 61–70,2006. 2, 6, 7
[11] Y. M. Kim, D. Chan, C. Theobalt, and S. Thrun. Design andcalibration of a multi-view TOF sensor fusion system. InComputer Vision and Pattern Recognition Workshops, 2008.CVPRW ’08. IEEE Computer Society Conference on, pages1 –7, Jun. 2008. 2
[12] M. Maruyama and S. Abe. Range sensing by projecting mul-tiple slits with random cuts. In SPIE Optics, Illumination,and Image Sensing for Machine Vision IV, volume 1194,pages 216–224, 1989. 2
[13] T. Matsuyama, X. Wu, T. Takai, and S. Nobuhara. Real-time 3D shape reconstruction, dynamic 3D mesh deforma-tion, and high fidelity visualization for 3D video. Comput.Vis. Image Underst., 96(3):393–434, Dec. 2004. 1
[15] M. Nakazawa, I. Mitsugami, Y. Makihara, H. Nakajima,H. Yamazoe, H. Habe, and Y. Yagi. Dynamic scene recon-struction using asynchronous multiple kinects. In The 7thInt. Workshop on Robust Computer Vision (IWRCV2013),Jan. 2013. 2
[16] R. Sagawa, Y. Ota, Y. Yagi, R. Furukawa, N. Asada, andH. Kawasaki. Dense 3d reconstruction method using a singlepattern for fast moving object. In ICCV, 2009. 2, 3
[17] R. Sagawa, K. Sakashita, N. Kasuya, H. Kawasaki, R. Fu-rukawa, and Y. Yagi. Grid-based active stereo with single-colored wave pattern for dense one-shot 3D scan. In Proc.2012 Second Joint 3DIM/3DPVT Conference, pages 363–370, Zurich, Switzerland, Oct. 2012. 2, 3, 4
[18] J. Tajima and M. Iwakawa. 3-D data acquisition by rainbowrange finder. In ICPR, pages 309–313, 1990. 2
[19] A. O. Ulusoy, F. Calakli, and G. Taubin. One-shot scanningusing de bruijn spaced grids. In The 7th IEEE Conf. 3DIM,pages 1786–1792, 2009. 2
[20] D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popovi,S. Rusinkiewicz, and W. Matusik. Dynamic shape captureusing multi-view photometric stereo. ACM Trans. Graphics(Proc. SIGGRAPH Asia), 28(5), 2009. 1, 2
[21] S. Zhang and P. Huang. High-resolution, real-time 3D shapeacquisition. In Proc. Conference on Computer Vision andPattern Recognition Workshop, page 28, 2004. 2