1/12 Vision based rock, paper, scissors game Patrik Malm Standa Mikeš József Németh István Vincze.

1/12

Vision based rock, paper, scissors game

Patrik Malm

Standa Mikeš

József Németh

István Vincze

2/12

Task

Create a vision based rock, paper, scissor gameRealtime webcam image-processing

3/12

Flowchart

4/12

Initiation

Smooth the image

Color-space conversion - L*u*v

The background is captured so that it can be subtracted in the following segmentation

5/12

Hand segmentation

Normal Bayesian classifier

Mathematical morphology to decrease noise

Use labeling to find the largest coherent component (or the two largest components in the 2 player game)

Cut the hand at the wrist

6/12

Isolating the handCut the arm where the size is 70% of the maximum projection valueIt works best for arms with vertical orientationImproved solution using PCA which works for rotated arms as well

7/12

Isolating the hand

The approximation of hand rotation is done using principal component analysis (PCA)It is computed using eigenvectors of the second-order moment matrix (covariance matrix)

8/12

Hand tracking

Find the centre point of the hand in each frame

Analyze the movement by looking at the relative change in position over a few frames

The hand tracking is used to countdown

9/12

Gesture identification

Logarithmic values of three of the Hu set of invariant moments

Bayesian classifier which then determines the current hand gesture

10/12

Computer player

Computer choose randomly (current)

Markov chain to recognize the human opponent’s game strategy (implemented but not added)

11/12

Known issues

Skin color variationsSensitivity to lightingBackground artifacts (skin or skin-like colored objects)

12/12

Thank you for your attention

Bibliography

A.A. Argyros, M.I.A. Lourakis, Real time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera, in proceedings of the European Conference on Computer Vision (ECCV 04), Springer-Verlag, vol. 3, pp. 368-379, Prague, Chech Republic,May,11-14,2004.

OpenCV Library Wiki, "http://opencvlibrary.sourceforge.net/".