Teleimmersion

Internet-Wide Multi-Party Tele-Immersion Framework

for

Remote 3D Collaboration

RINU MARKS.PREG [email protected]

Tele-Immersion is a new technology that enables a user to share a virtual space with remote participants.

WHAT IS TELE-IMMERSION ???

Existing System

• Video conferencing.• Fails to keep eye to eye contact.• User interaction is limitted.• When no. of persons conferencing

increases it will be hard to concentrate on person who is actually talking.

site 3site 1

site 2

BASIC CONCEPT

• Not merely the 3D reproduction of the human body.

• It also allows us to interact with virtual objects in the shared virtual environment

Interaction with virtual objects

• Tele-immersion reconstructs the objects in the real world in real-time, and mixes the real objects with the virtual environment

Remote Controller Function(wii)

• Various operations on 3d virtual objects such as “move”, “rotate”, “select”, “change texture”, “expand”, “shrink”, “highlight”

• Helps in eye to eye communication in multi party enironment.

ARRANGEMENTS

ARRANGEMENTS

•B&W cameras are for stereo construction.•The color camera is for extracting the appearance

ARRANGEMENT

3D IMAGE CONSTRUCTION

3D image construction

Stages in 3D conversion

• Acquisition stage• Reconstruction stage

Acquisition stage

• capture a variable amount of camera feeds .

• image-based calculations and color silhouette extraction.

• Image compression.

Sterio based reconstruction stage

Sterio based reconstruction1. Calculate initial matching cost

• and are the color intensities in the left and right images

• d is the disparity hypothesis.• denotes pixel in the left image

and denotes pixel in the right image

2. Construct a square window of predefined size with p as centre.

3. For each pixel q in window calculate adaptive support weight.

• and denotes the color and euclidean distance between p and q.

• are constants.

4. Calculate aggregated cost of each support window.

5. By using optimization technique we calculate depth of discontinuous areas and reconstruct the body.

The user whole body is extracted by taking the difference between the current video frame and the registered background image.

BACK GROUND SUBTRACTION

Real time compression

• Reconstructed 3d model is very large in size.

• Real-time compression is essential for real-time delivery.

• Modified jpeg compression.

JPEG COMPRESSION

Each block is defined as M[x,y] (i -1) * d <=x < i * d , 0 <= i < w ( j -1) * d <= y < j * d , 0 <=j < h d is the size w width and h height.

For each M[i , j]

• If all M[x, y] == 0 , then M '[i, j] = N .

• If all M[x, y] == 1, then M '[i, j] = A .

• If exists any M[x, y] == 0 and any M[x, y] == 1 , then M '[i, j] = B

Multi-Party Tele-Immersion using internet

Server side

• The TI server provides a virtual environment and room model to divide the users into groups.

• The system information such as user login/logout and their manipulations are written in database.

• The portal provides user login and real-time status display through web browsers with the help of database

Communication model

Hardware Requirements

• 4-core Intel Xeon W3530 processor

• 6GB DDR3 RAM• 2*500GB 7200rpm SATA Hard disk• NVIDIA Geforce GTX 470 display

card

Time delays

Time cost of 3D reconstruction & encoding

Time cost of decoding & rendering

Remote interaction delay on 3D objects

Bandwidth requirement

Applications

• Remote medical diagnosis and therapy.

• Remote and distributed education.• Distributed artistic performance and

interaction.• Virtual travel and tourism.• Automobile Engineering

Challenges

• Quality of transmission affected by high packet loss

• Very Expensive• High bandwidth requirement.• High speed internet is required

(min of 60 MBPS)

CONCLUSION

• Still in its early stages of development.

• Challenges are need to be met.• 3d reconstruction technology

needs to be improved since the reproduced images are jittering.

• Tele immersion technology can surely lead to great developments in the field of communication.

References

• Proposal of Multimodal Communications System Using Tele-Immersion Technology ; IEEE Conference

• Tele-Immersion Environment for Video Avatar Based CVE ; IEEE Conference

• Distance Learning in Tele-Immersion Environment; IEEE Conference

• http://tele-immersion.citris-uc.org• http://graphics.cs.brown.edu/research/telei

THANK YOU