Virtual painting project

Virtual painting project

By: By: Leetal Gruper Leetal Gruper Tsafrir KameloTsafrir Kamelo

Supervisor: Michael KolomenkinSupervisor: Michael KolomenkinAdvisor from 3DV systems: Sagi Katz Advisor from 3DV systems: Sagi Katz

• The following project introduces a new type of interface based on the user's hand and body movements as captured by the novel 3DV systems Zcam™

Introduction

Project goal

• User will be able to paint on a virtual surface

• The program is intended for small children

• The user will stand in front of the camera, paint with any part of their body they wish, and see the painting on the computer screen

System description

• The system consists of the 3DV systems Zcam™ and a program that runs on windows

• The program includes inside:– Ogre – a 3D engine.– Depth camera interface.– Upper body tracking interface.

The virtual paint softwareThe virtual paint software

The program flow

• The user stands in front of the Zcam™

• Selects a color to paint

• Starts painting with every part of his body that cross the depth threshold

The program

• Starts with a configuration dialog to setup

Modes of operation

• Painting mode• Color selection mode:

– Red, green and blue– Cyan, magenta and yellow– White, orange and brown

• The user can paint in both modes. The only difference is in the interface presented on the screen

First color selection screenFirst color selection screen

Second color selection screenSecond color selection screen

Third color selection screenThird color selection screen

Passing between modes

• To move from color selection mode to the painting mode, one needs only to select a color or the eraser

• To move from the painting mode to the color selection mode, one should just press the arrow button

• To move between different color options, one needs to press the 2 arrows appearing on each side

Press the

Fine resolution painting

• An entire hand is to crude for fine painting

• Need for doing some pre-processing on the input depth image

Fine resolution painting cont.

• The algorithm we finally used was based on the assumption that we mainly paint homogenous objects

• The number of layers removed (which translate to a more or less finer resolution) is controlled from the keyboard numbers. 9 is the finest resolution possible and 1 is not removing layers at all

The algorithm

1. Scan the frame in an interlace order

2. For every pixel in depth range 1. Has at least one pixel from 4 surrounding

neighbors that are at a certain distance that isn't in our depth range? 1. Don’t paint the pixel

2. Else1. Paint the pixel

Conclusions

• We created software that uses an RGBD camera as an input interface and creates a fun game intended for children

• This project was our first encounter with computer graphics

• We have acquired knowledge in basic computer graphics • We have also learned how to use a rendering platform

as OGRE to create all our graphical needs• For the students that will follow us and want to use

OGRE, we have created a tutorial comprising all our knowledge

Thank you Michael and SagiThank you Michael and Sagifor all your help and guidancefor all your help and guidance