MACHINE VISION GROUP Multimodal sensing-based camera applications Miguel Bordallo 1 , Jari Hannuksela 1 , Olli Silvén 1 and Markku Vehviläinen 2 1 University of Oulu, Finland 2 Nokia Research Center, Tampere, Finland Jari Hannuksela, Olli Silvén Machine Vision Group, Infotech Oulu Department of Electrical and Information Engineeering University of Oulu, Finland
19
Embed
MACHINE VISION GROUP Multimodal sensing-based camera applications Miguel Bordallo 1, Jari Hannuksela 1, Olli Silvén 1 and Markku Vehviläinen 2 1 University.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
MACHINE VISION GROUP
Multimodal sensing-based camera applications
Miguel Bordallo1, Jari Hannuksela1, Olli Silvén1 and Markku Vehviläinen2
1 University of Oulu, Finland2 Nokia Research Center, Tampere, Finland
Jari Hannuksela, Olli SilvénMachine Vision Group, Infotech Oulu
Department of Electrical and Information EngineeeringUniversity of Oulu, Finland
MACHINE VISION GROUP
Outline
Introduction• Modern movile device with multiple
sensorsVision-based User InterfacesSensor data fusion systemApplication case implementations
• Video analysis detects ego-movements and analyzes the context
• Accelerometers provide complementary motion data
MACHINE VISION GROUP
Video analysis
- Every frame divided into regions- Selection of feature blocks- Estimation of block displacements- Analysis of uncertainty
- Results: 4-paramenter model- X, Y, Z, r
MACHINE VISION GROUP
Sensor data fusion: Accelerometers
MACHINE VISION GROUP
Sensor data fusion
Model the device movement with the folowing
Define a state vector: position, speed, acceleration
Define a measurement model
Apply Kalman filtering adding accelerometer values: State prediction + state correction
MACHINE VISION GROUP
Application cases
• Sensor data fusion method applied in two applications– Implemented on a Nokia N900 mobile phone
• Motion based image browser– Allows browsing large images and maps with one hand operation– Works under different light conditions
• Sensor assisted panorama imaging– Stitches panorama images in real time from video frames– Increased robustness against fast movements and no-texture
frames
MACHINE VISION GROUP
Motion based image browser
Uses fusion model from accelerometers + video analysis to generate commands
• Scroll up/down/left/right• Zoom in/out
Light sensor decides:• if camera should be turned on • weighting factors and uncertainties• 3 modes defined:
• Good image quality (video analysis + accelerometer correction)• Bad image (accelerometers have increased contribution)• No image (only accelerometers are used)
•Uses sensor fusion model to compute camera motion•Increased robustness against fast movements and frames with low/smooth texture
Registration
MACHINE VISION GROUP
Panorama: Sensor uses II
•Uses accelerometer data to detect blur•Detects unwanted shake/tilt•Integrated in scoring system
Selection
MACHINE VISION GROUP
Summary
• Vision based interfaces offer high interactivity with one hand operation
• They present several limitations• Sensor fusion improves motion estimation
adding robusness against fast movements and dark conditions
• The framework can be included in several applications (e.g. as a part of Motion Estimation API)
MACHINE VISION GROUP
Conclusions
• We have presented a sensor fusion framework that fuses vide analysis with motion sensors (acelerometers+magnetometers+gyroscopes)
• We have presented two applications cases that make use of sensor data fusion and integration
• The applications presented are by no means the only ways to apply vision or multiple sensors, and one may find new interesting possibilities in further research