Construction of General HMMs from a Few Hand Motions for Sign Language Word Recognition Stop Raise right Lower right Stop Stop Stop Raise right Lower right Stop S 1 S 2 S 3 S 4 S 5 Stop Raise right Lower right Stop Stop Stop Raise right Lower right Stop S 1 S 2 S 3 S 4 S 5 th virtual samples Various training samples The HMM with highest likelihood for training samples High likelihood Low likelihood Few training samples eal samples Virtual samples High likelihood Tadashi Matsuo, Yoshiaki Shirai, Nobutaka Shimada Ritsumeikan University/Department of Human and Computer Intelligence, Shiga, Japan 1. Recognition with HMM Take the model with the largest likelihood Calculate likelihood Extract feature Raise Both hands Spread hands Stop Lower hands Model for word 1 Raise Right hand Lower Right hand Stop Recognition Result Input Images Numeric features Model for word 2 Model for word 3 . What is a problem? Motions for the same word may differ in hand shape, speed, track, etc. We generate many candidate HMMs and evaluate them. How to select a HMM without over-fitting training samples? . Virtual samples They are desirable, but require high cost. They may cause a over-fitting HMM. Over-fitting can be avoided without high cost. The topology of HMM should reflect the acceptable variation of the word. An input motion accepta ble but differe nt from trainin g samples generat ive model
2
Embed
Construction of General HMMs from a Few Hand Motions for Sign Language Word Recognition
Construction of General HMMs from a Few Hand Motions for Sign Language Word Recognition. Tadashi Matsuo, Yoshiaki Shirai , Nobutaka Shimada Ritsumeikan University/Department of Human and Computer Intelligence, Shiga, Japan. Calculate likelihood. 1. Recognition with HMM. - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Construction of General HMMs from a Few Hand Motions for Sign Language Word Recognition
Stop Raiseright
LowerrightStop Stop
Stop Raise right Lower right Stop
S1 S2 S3 S4 S5
Stop Raiseright
LowerrightStop Stop
Stop Raise right Lower right Stop
S1 S2
S3S4 S5
With virtual samples
Various training samples
The HMM with highest likelihood for training samples
High likelihood
Low likelihood
Few training samples
Real samples Virtual samples
High likelihood
Tadashi Matsuo, Yoshiaki Shirai, Nobutaka Shimada Ritsumeikan University/Department of Human and Computer Intelligence, Shiga, Japan
1. Recognition with HMMTake the model with the
largest likelihood
Calculate likelihood
Extract feature
RaiseBoth hands
Spreadhands Stop Lower
hands
Model for word 1RaiseRight hand
LowerRight hand Stop
RecognitionResult
Input Images Numeric features
Model for word 2
Model for word 3
2. What is a problem?Motions for the same word may differ in hand shape, speed, track, etc.
We generate many candidate HMMs and evaluate them.
How to select a HMM without over-fitting training samples?
3. Virtual samples
They are desirable, but require high cost.
They may cause a over-fitting HMM.
Over-fitting can be avoided without high cost.
The topology of HMM should reflect the acceptable variation of the word.
An input motion acceptable but different from training samples
generative model
Real samples Candidate HMMs
Virtual samples
Select the HMM with the highest likelihood
Stop Raiseright
Lowerright StopStop
Stop Raiseright
Lowerright StopStop
Stop Raiseright
Lowerright Stop
Stop Raiseright
Lowerright
Rotateright
Groups of segmented real samples[1] HMM for generating
virtual samples
HMM for generating virtual samples
4. How to generate virtual samples
5. Total system
Virtual samples
Each virtual sample is a variation of one of the groups.
Tab.1 Recognition accuracy for a speaker not used when training HMMs20 words, 3 person, 3 motions for a person and a word
Virtual samples improve HMM selection. Over-fitting can be avoided without collecting high cost real samples.
[1]T. Matsuo, Y. Shirai, N. Shimada, "Automatic Generation of HMM Topology for Sign Language Recognition”, The 19th International Conference on PATTERN RECOGNITION (ICPR2008), (2008).
Generate virtual samples
7. Conclusion
They are generated by integration of motion segments[1].