1 3D Tele-Collaboration 3D Tele-Collaboration over Internet2 over Internet2 Herman Towles, UNC-CH Herman Towles, UNC-CH representing members of the representing members of the National Tele-Immersion Initiative (NTII) National Tele-Immersion Initiative (NTII) ITP 2002 ITP 2002 Juan-les-Pins, France Juan-les-Pins, France 06 December 2002 06 December 2002
26
Embed
1 3D Tele-Collaboration over Internet2 Herman Towles, UNC-CH representing members of the National Tele-Immersion Initiative (NTII) ITP 2002 Juan-les-Pins,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
3D Tele-Collaboration3D Tele-Collaboration
over Internet2over Internet2
Herman Towles, UNC-CH Herman Towles, UNC-CH
representing members of therepresenting members of the
National Tele-Immersion Initiative (NTII)National Tele-Immersion Initiative (NTII)
ITP 2002ITP 2002
Juan-les-Pins, FranceJuan-les-Pins, France
06 December 200206 December 2002
3D Tele-Collaboration3D Tele-Collaboration
over Internet2over Internet2
Herman Towles, UNC-CH Herman Towles, UNC-CH
representing members of therepresenting members of the
National Tele-Immersion Initiative (NTII)National Tele-Immersion Initiative (NTII)
• University of North Carolina at Chapel University of North Carolina at Chapel HillHill– Wei-Chao Chen, Ruigang Yang, Sang-Uok Kum, and Henry FuchsWei-Chao Chen, Ruigang Yang, Sang-Uok Kum, and Henry Fuchs
• University of PennsylvaniaUniversity of Pennsylvania– Nikhil Kelshikar, Jane Mulligan, and Kostas DaniilidisNikhil Kelshikar, Jane Mulligan, and Kostas Daniilidis
• Brown UniversityBrown University– Loring Holden, Bob Zeleznik, and Andy Van DamLoring Holden, Bob Zeleznik, and Andy Van Dam
• Advanced Network & ServicesAdvanced Network & Services– Amela Sadagic and Jaron LanierAmela Sadagic and Jaron Lanier
• University of North Carolina at Chapel University of North Carolina at Chapel HillHill– Wei-Chao Chen, Ruigang Yang, Sang-Uok Kum, and Henry FuchsWei-Chao Chen, Ruigang Yang, Sang-Uok Kum, and Henry Fuchs
• University of PennsylvaniaUniversity of Pennsylvania– Nikhil Kelshikar, Jane Mulligan, and Kostas DaniilidisNikhil Kelshikar, Jane Mulligan, and Kostas Daniilidis
• Brown UniversityBrown University– Loring Holden, Bob Zeleznik, and Andy Van DamLoring Holden, Bob Zeleznik, and Andy Van Dam
• Advanced Network & ServicesAdvanced Network & Services– Amela Sadagic and Jaron LanierAmela Sadagic and Jaron Lanier
3
Clear Motivation to Provide Clear Motivation to Provide
• Higher ResolutionHigher Resolution
• Larger, more immersive Field-of-ViewLarger, more immersive Field-of-View
• Participants at Accurate Geometric ScaleParticipants at Accurate Geometric Scale
– HiBall to Eyeball HiBall to Eyeball CalibrationCalibration
• PC Network Server PC Network Server
17
Rendering Configurations Rendering Configurations
• One PC Configuration (Linux)One PC Configuration (Linux)– Dual-channel NVIDIA graphicsDual-channel NVIDIA graphics
• Three PC Configuration (Linux)Three PC Configuration (Linux)– Separate left & right-eye rendering PCs w/NVIDIA graphicsSeparate left & right-eye rendering PCs w/NVIDIA graphics
– One PC used as network interface, multicasts depth map One PC used as network interface, multicasts depth map stream to rendering PCsstream to rendering PCs
– Asynchronous updates of 3D Reconstruction (2-3Hz) & Scene Asynchronous updates of 3D Reconstruction (2-3Hz) & Scene Graph (20Hz)Graph (20Hz)
• Newest Rendering Configuration 10-20XNewest Rendering Configuration 10-20X– 2.4GHz, GeForce4, Multi-Threaded, VAR Arrays2.4GHz, GeForce4, Multi-Threaded, VAR Arrays
• One PC Configuration (Linux)One PC Configuration (Linux)– Dual-channel NVIDIA graphicsDual-channel NVIDIA graphics
• Three PC Configuration (Linux)Three PC Configuration (Linux)– Separate left & right-eye rendering PCs w/NVIDIA graphicsSeparate left & right-eye rendering PCs w/NVIDIA graphics
– One PC used as network interface, multicasts depth map One PC used as network interface, multicasts depth map stream to rendering PCsstream to rendering PCs
– Asynchronous updates of 3D Reconstruction (2-3Hz) & Scene Asynchronous updates of 3D Reconstruction (2-3Hz) & Scene Graph (20Hz)Graph (20Hz)
• Newest Rendering Configuration 10-20XNewest Rendering Configuration 10-20X– 2.4GHz, GeForce4, Multi-Threaded, VAR Arrays2.4GHz, GeForce4, Multi-Threaded, VAR Arrays
18
Network Considerations Network Considerations
• All Tests over Internet2All Tests over Internet2
• Data Rates of ~20-75 Mbps from Data Rates of ~20-75 Mbps from Armonk, NY and Philadelphia into Armonk, NY and Philadelphia into Chapel HillChapel Hill
– 320 x 240 Resolution320 x 240 Resolution
– Up to 5 Reconstruction Views per siteUp to 5 Reconstruction Views per site
– Frame Rates 2-3 fpsFrame Rates 2-3 fps
• TCP/IP TCP/IP
• Latency of 2-3 seconds typicalLatency of 2-3 seconds typical
• All Tests over Internet2All Tests over Internet2
• Data Rates of ~20-75 Mbps from Data Rates of ~20-75 Mbps from Armonk, NY and Philadelphia into Armonk, NY and Philadelphia into Chapel HillChapel Hill
– 320 x 240 Resolution320 x 240 Resolution
– Up to 5 Reconstruction Views per siteUp to 5 Reconstruction Views per site
– Frame Rates 2-3 fpsFrame Rates 2-3 fps
• TCP/IP TCP/IP
• Latency of 2-3 seconds typicalLatency of 2-3 seconds typical
19
Presentation Outline Presentation Outline
• Motivation and Related WorkMotivation and Related Work
• NTII Tele-Collaboration TestbedNTII Tele-Collaboration Testbed– Acquisition and 3D ReconstructionAcquisition and 3D Reconstruction
– Collaborative Graphics & User InterfacesCollaborative Graphics & User Interfaces
– Rendering & DisplayRendering & Display
– Network Network
• ResultsResults
• Future ChallengesFuture Challenges
• Motivation and Related WorkMotivation and Related Work
• NTII Tele-Collaboration TestbedNTII Tele-Collaboration Testbed– Acquisition and 3D ReconstructionAcquisition and 3D Reconstruction
– Collaborative Graphics & User InterfacesCollaborative Graphics & User Interfaces
– Rendering & DisplayRendering & Display
– Network Network
• ResultsResults
• Future ChallengesFuture Challenges
20
Results Results
‘‘Roll the Tape’Roll the Tape’‘‘Roll the Tape’Roll the Tape’
21
SummarySummary
• ‘‘One-on-One’ 3D Tele-Immersion One-on-One’ 3D Tele-Immersion TestbedTestbed
• Operation over Internet2 between Operation over Internet2 between Chapel Hill, Philadelphia and ArmonkChapel Hill, Philadelphia and Armonk
• Audio over H.323 or POTSAudio over H.323 or POTS
22
Future ChallengesFuture Challenges
• Improved 3D Reconstruction QualityImproved 3D Reconstruction Quality– Larger Working Volume, Faster Frame Rates – 60 camerasLarger Working Volume, Faster Frame Rates – 60 cameras
– Fewer Reconstruction Errors (using structured light and Fewer Reconstruction Errors (using structured light and adaptive correlation kernels)adaptive correlation kernels)
• Reduce System Latency and Reduce System Latency and Susceptibility to Network CongestionSusceptibility to Network Congestion– Pipelined architecturePipelined architecture
– Shunt Protocol (between TCP/UDP and IP layers) that Shunt Protocol (between TCP/UDP and IP layers) that allows multiple flows to do coordinated congestion controlallows multiple flows to do coordinated congestion control
• Full Duplex OperationFull Duplex Operation
• Unobtrusive OperationUnobtrusive Operation– No headmounts, No eyeglasses!No headmounts, No eyeglasses!
• Improved 3D Reconstruction QualityImproved 3D Reconstruction Quality– Larger Working Volume, Faster Frame Rates – 60 camerasLarger Working Volume, Faster Frame Rates – 60 cameras
– Fewer Reconstruction Errors (using structured light and Fewer Reconstruction Errors (using structured light and adaptive correlation kernels)adaptive correlation kernels)
• Reduce System Latency and Reduce System Latency and Susceptibility to Network CongestionSusceptibility to Network Congestion– Pipelined architecturePipelined architecture
– Shunt Protocol (between TCP/UDP and IP layers) that Shunt Protocol (between TCP/UDP and IP layers) that allows multiple flows to do coordinated congestion controlallows multiple flows to do coordinated congestion control
• Full Duplex OperationFull Duplex Operation
• Unobtrusive OperationUnobtrusive Operation– No headmounts, No eyeglasses!No headmounts, No eyeglasses!
23
Thank You Thank You
Research funded byResearch funded byAdvanced Network and Services, Inc. and Advanced Network and Services, Inc. and
National Science Foundation (USA)National Science Foundation (USA)
Research funded byResearch funded byAdvanced Network and Services, Inc. and Advanced Network and Services, Inc. and
National Science Foundation (USA)National Science Foundation (USA)
24
UPenn Acquisition ArrayUPenn Acquisition Array
Fifteen Sony Digital 1394 Cameras – Five Trinocular Views