Top Banner
Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006
15

Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Dec 19, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality

Jackie Zhu

Oct. 24, 2006

Page 2: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Outline

IntroductionExtended Active Appearance Models

(AAMs) FittingAlgorithm

Offline construction of 3D shape model Estimate 3D pose and non-rigid shape parameters

Experiemental ResultsConclusion

Page 3: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Backgroud

Rigid Object L. Vacchetti et al. (PAMI‘04) proposed an efficient

solution for 3D rigid object tracking Two 2D AAMs approach for rigid object pose

estimation

Non-rigid Object V.Blandz: 3D Morphable Models J.Ahlberg: 3D AAM with generic Model. Jing X. (CVPR‘05) 2D+3D AAM

Page 4: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Overview

Page 5: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Extended AAMs Fitting Algorithm

Page 6: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

AAM Fitting Sample

The AAMs are built up with 140 still face image belonging to 20 individuals, 7 images for each. The fitting experiment is performed on an AAM with 14 shape parameters, 68 texture parameters, and 36335 color pixels.

Page 7: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Algorithm

Building offline basis: Acquire the 2D shape of objects using the AAM fitting algorithm, then construct the 3D shape basis.

Online tracking: Estimate the 3D pose and shape parameters simultaneously via local bundle adjustment by building up the point correspondences between 2D and 3D.

Page 8: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Algorithm: Offline 3D Model

Page 9: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Algorithm: Online Pose Estimation

The optimization problem can be derived as:

Where

Page 10: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Experiemental Results I

Page 11: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Experimental Results II

Page 12: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Experimental Results

Page 13: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

Conclusion

A novel two-stage scheme for online non-rigid shape recovery toward Augmented Reality applications using AAMs.

Obtain unbroken point correspondences across multiple frames to construct 3D shape models

Provide 2D to 3D vertex correspondences in the online tracking.

An efficient algorithm is proposed to estimate both 3D pose and non-rigid shape parameters via local bundle adjustment.

Page 14: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

References:

Ahlberg, J.: Using the active appearance algorithm for face and facial feature tracking. In: Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001. Proceedings. IEEE ICCV Workshop on. (2001) 68-72

Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2d+3d active appearance models. In: IEEE CVPR'2004. Volume 2. (2004) 535-542

Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. PAMI 26 (2004)

Cootes, T., Edwards, G., Taylo, C.: Active appearance models. IEEE Trans. PAMI 23 (2001)

C.Bregler, A.Hertzmann, H.Biermann: Recovering non-rigid 3d shape from image streams. In: IEEE CVPR'2000. Volume 2. (2000) 690-696

Jianke Zhu, Steven C.H. Hoi and Michael R. Lyu: The Real-Time Non-Rigid Shape Recovery via Active Appearance Models for Augmented Reality. In: ECCV'2006. Volume 1. LNCS 3951. (2006) 186-197.

Page 15: Real-Time Non-Rigid Shape Recovery via AAMs for Augmented Reality Jackie Zhu Oct. 24, 2006.

The End

Thank You !