NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Proposed Collaboration on Access Proposed Collaboration on Access Grid / Ubiquitous ComputingGrid / Ubiquitous Computing
JongWon Kim and Juwon Park
The Temporary Genkai-Hyeonhae Meeting
Feb. 27th, 2003
Networked Media LaboratoryDept. of Information & Communications
Kwang-Ju Institute of Science & Technology (K-JIST)[email protected]
http://netmedia.kjist.ac.kr
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ContentsContents
Collaboration on Access Grid
Collaboration on Ubiquitous Computing
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration on
Access Grid
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Access GridAccess Grid
Access GRIDoSupport group-to-group interaction across the Grid
Components of an AG NodeComponents of an AG Node
Digital Video
Digital Video
Digital Audio
NETWORK
MixerControl Computer
NTSC Video
RGB Video
Analog Audio
Video Capture
Computer
DisplayComputer
AudioCapture
Computer
EchoCanceller
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Usage AreasUsage Areas The Academic and Research, Government, Private Sectors Making Remote Collaborations Work across Boundaries Check “Multi-Sector Collaboration over the Access Grid” b
y J. T. von Hoffman (Boston Univ)”
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration Issue for H-G Joint ProjectsCollaboration Issue for H-G Joint Projects Access Grid (#4) Status 2 Contact: Koji Okamura (Kyushu Univ.) JongWon Kim (K-JIST)
ParticipantsKyushu Nat`l Univ, AISTKISTI, K-JIST, Chonnam U, HYU
Description (to be revised)The Access Grid, as a communication channel for users from remote sites, can give researc
hers and scientists in both Korea and Japan a seamless and immersive collaboration environment. With this project, we want to explore the potential of Access Grid in supporting collaborative research between two countries. To realize it, we will collaborate to research ways to build low-cost Access Grid system (e.g. mini-AG or PIG) and to promote its deployment. Also, we will conduct a joint research to improve current AG systems to better support enhanced video/audio quality, interaction, security, and inter-operablity.
Expected Results (to be revised) - Provide a seamless colloaboration channel for researchers and scientists. - Promote the adoption of multicast technology over H-G link.
Required bandwidth - Dependent on the number of site, camera per site, video rates (normally full access node requires bandwidth around 1.5Mbps) - Multicast support
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Relation with AG and other projectsRelation with AG and other projects
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
N*Grid Project lead by KISTIN*Grid Project lead by KISTI
APEC APGrid
APAN APGrid
GGF Grid
Euro APGrid
ANL/NCSA
Grid Tech-center GNOC
N BT Grid
N NT Grid
N CT Grid
N ET Grid
N IT Grid
N ST Grid
N Supercmpt Grid
N Clusters Grid
N Appls. Grid
Access Grid
VR Grid
Security
Resource Mngt.
Data Mngt.
Inf. Service
National Grid Infra Grid Standard Grid Middleware Research GFK
N*GridKISTI(2002~2006)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
KISTI AGKISTI AG First AG in Korea (2002)
KISTI’s Access Grid Node
NetworkNetworkenabled enabled
IP IP multicastmulticast
NetworkNetworkenabled enabled
IP IP multicastmulticast
Audio CaptureComputer
Display Computer
Video CaptureComputer
Mixer
Echo Canceller
NTSC Video
AnalogAudio
Digital Video
Digital Video
Digital Audio
Shared App.Control
ControlComputer
RS232 Serial
RGB Video
Int. AG Community
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
K-JIST Mini AGK-JIST Mini AG Personal Interface to Access Grid (PIG) (Oct. 2002) With AG 1.2 release (note that PIG software is still very
delicate)
PIG @ K-JIST
AG Node @ KISTIKISTI
K-JIST
KIST
KIST-KISTI-K-JIST AG demo (2002.10.30)
1st AG Node in KOREA
1st PIG NodeIn KOREA
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Towards Enhanced Access Grid: e-AGTowards Enhanced Access Grid: e-AG An experimental, on-going trial to enhance the functionalities of
Access Grid by augmenting improved support for immersive 3D Display, Human Computer Interaction, and Networking.
With e-AG, we hope supporto All functionalities of Access Grid
o Stereo video delivery and display
o Collaborative use of 3D computer graphics.
Video CaptureComputer
Audio CaptureComputerControl
Computer Echo
DisplayComputer
Enhanced Partsilver
Network
3D
2D
Input DeviceComputer
Access Grid
screen
StereoCamera
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Immersiveness Enhancement for Access Grid via Stereo Immersiveness Enhancement for Access Grid via Stereo Video DeliveryVideo Delivery
For providing the immersiveness, we include the stereo video delivery functionality to EAG
Developing 3D video delivery module enables the reliable and real-time The developed system supports
o Reliable networking of real-time media with reliable transport via ARQ, FEC, and their hybrid
o Versatile display of stereo video in Left, Interlaced, Left & Right, and Above & Below views.
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Immersiveness EnhancementImmersiveness Enhancement for Access Grid via Computer Graphics for Access Grid via Computer Graphics
For the graphics side of 3D, EAG is being enhanced by supporting CG-based sharing in various communication architectures.
Combination of push repository in server memory and file repository method is implemented to support multi-user collaborative environment.
UIC EVL’s QUANTA is adopted as a baseline networking and data management module
VR software tools such as CAVELib or VR Juggler are also selected and utilized to build immersive 3D display.
Shared Memory
Server
Client A (VR Juggler)
3D Display Input
Networking(Shared Memory Client)
Client B (VR Juggler)
3D Display Input
Networking(Shared Memory Client)
Client C (CAVELib)
3D Display Input
Networking(Shared Memory Client)
Network
CMD, ID, Coord. CMD, ID, Coord. CMD, ID, Coord.
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration Issue for H-G Joint ProjectsCollaboration Issue for H-G Joint Projects
On-line Joint Technical Conference on Access Grid Joint Project (Long-term)
o Joint participation in AG 2.0 and beyond
Ohmuta
Kurume
NogataIizukaTagawa
Fukuoka
Kitakyushu
Kitakyushu CityHigh-speed LAN
QGPOP (ISIT)
To Tokyo
Research Network in KoreaKorean Partner CarrierKJCN Japanese Partner CarriersFukuoka Gigabit HighwayGENKAI ProjectQGPOP partKJCN
Korea
Busan
KISTI
KIST
GENKAI/Hyeonhae Project
APANInternet2
GbE
2.4Gbps150Mbps
ShimonosekiUniv. of Fishery
FGH AP
Japan
Submarine cable landing stationNetwork equipment (Routers etc.)
250 km
RepeaterlessKJIST
Gwangju
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration on
Ubiquitous Computing
- Seamless Mobile Service? -
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
What’s UbiComp?What’s UbiComp?
What’s Ubiquitous Computing (UbiComp)?
Distributed Systems
Distributed Systems
Mobile Computing
Mobile Computing
Ubiquitous ComputingUbiquitous Computing++ ++
•Mobile networking: mobile IP, ad hoc protocol, etc.•Mobile information access: BW-adaptive, data consistency, etc.•Energy-aware systems: process scheduling, memory management, etc.•Location-sensitive systems: location sensing, location-based services, etc.
•Smart space: context-aware•Invisible: smart agent•Localized scalability: physical distance-based•Uneven condition: heterogeneous infrastructure
•Remote communication•Fault tolerance•High availability•Remote info access
Courtesy of K-JIST U-VR Lab
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
UbiComp Challenges: ubiquitous NetwoUbiComp Challenges: ubiquitous Networkingrking
Infrastructure: ubi-networkingo Beyond desktops/servers
o Connected to networks
Networked appliances Sensors Historical sites & other locations
Embedded Mobile
Interactive
WearableActive badge
Location
SensorFlight
Simulator
Network
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration Issue for H-G Joint ProjectsCollaboration Issue for H-G Joint Projects Ubiquitous (Wireline/wireless) Computing (#18) Status 2 Contact: Morioka ([email protected]) JW Kim ([email protected])
Participants ISIT KJIST
Description We are planning to build a prototype version of ubiquitous environment utilizing Mobile IP and IEEE 802.11a/b (and other L2 media) connecting Fukuoka, JAPAN and Gw
angju, KOREA. On top of the built ubiquitous environment we will explore example applications such as u
ser tracking and IP mobile video streaming.
Expected Result In our joint effort, instead of trying to build hot spots, we will focus on building real ubiqui
tous environement with mobility and security support. We hope to let people in both countries to better locate each other and communicate ubiquitously.
Bandwidth Peak 100 Mbps / Average 10 Mbps
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Project Scope & ObjectiveProject Scope & Objective Mobile Media Delivery via Wireless Multicast and
Network Adaptation
Wireless Network Wireless Network AdaptationAdaptation
Wireless AwareWireless AwareMedia ApplicationMedia Application
Advanced Streaming Advanced Streaming (Wireless + Multicast)(Wireless + Multicast)
Group Group CommunicationCommunication(1-to-N, M-to-N)(1-to-N, M-to-N)
Mobile IP / Mobile IP / Wireless Wireless MulticastMulticast
Reliability:Better Quality
Efficiency: Better Bandwidth
Utilization
Reliable Reliable MulticastMulticast
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Project Scope & Objective (Cont.)Project Scope & Objective (Cont.) For Streaming Media Applications over Wired/Wireless
Integrated Networkso Real-time streaming of stored contents only (but can include on-
line streaming) to enable buffering (up to 3sec)o Maybe extended to conferencing (limited scope) at later phase
of project Will address the challenges such as
o Mobility: How to track user’s movement and sustain seamless streaming?
o Multicasting: How to enable sharing of wireless channel efficiently?
o End-to-end QoS: How to maintain the quality of media streaming over highly fluctuating wireless network?
Anticipated Duration: 2002. 12 ~ 2006.12 (4 years)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Implementation Phase I (2003): One-to-one Streaming Implementation Phase I (2003): One-to-one Streaming
Environmento Mobile IPv4o IEEE 802.11bo Server: Darwin or
Helix Servero Client – Notebooko MPEG-2/4 streaming
Objectiveso Implement the
streaming server/client that provides seamless streaming (i.e., no service disruption during handoff).
FAFAFAFA
Media Server
Adaptive streaming client
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Implementation Phase II (2004): One-to-many StreamingImplementation Phase II (2004): One-to-many Streaming
Environmento Mobile IPv6o IEEE 802.11b or
802.11ao Client Extension to
PDA?o MPEG-2/4 streamingo Source Specific
Multicast (SSM)
Objectiveso Implement the
streaming server/client providing that provides seamless multicast streaming.
FAFAFAFA
Media Server
Adaptive streaming client
Multicast
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Mobility Challenge: Mobile IPMobility Challenge: Mobile IP
Smooth and Fast Handoff ?
6
7
5
FAold
CN
FAnew
2
3
4
Output buffer
Handoff
2
3
4
Forwarding buffer
to FAnew
2
3
4
Bind
forwarding
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Mobility Challenge: Handoff-aware Mobility Challenge: Handoff-aware Streaming ClientStreaming Client
Adaptive buffer controlo To reduce the discontinuity of streamingo Increasing the buffer level of client before handoff
Questiono What is a proper buffer level?o What is a handoff duration?
Handoff is initiated
Handoff is completed
Bu
ffer
Leve
l
time
No playback
Handoff is initiated
Handoff is completed
Bu
ffer
Leve
l
time
< No buffer level control > < Buffer level control >
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Mobility Challenge: Handoff-aware Mobility Challenge: Handoff-aware Streaming ClientStreaming Client (Cont.) (Cont.)
Estimation of handoff duration by adopting K-JIST’s Transient Time Analysiso Source traffic rate
o Handoff load
o Background traffic load
o Beacon time period (router advertisement)
Buffer level controlo Using the handoff duration (HTP) to c
alculate the proper buffer level.
o Minimizing the buffer level
o Providing no media discontinuity
No service disruption during handoff
Collecting parameters
CalculatingHTP
EstimatingBuffer Level
Bc<Bt
IncreaseBuffer
DecreaseBuffer
Bc: Current Buffer LevelBt: Target Buffer Level
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
K-JIST Transient Time Analysis: How long K-JIST Transient Time Analysis: How long time does Handoff take?time does Handoff take?
Handoff Transient Time
Case 2
Case 3
Case 1
New streamOld stream
STP
STPUTP
STP UTP
HandoffInitialization
HTP
HTP
HTP
Source Traffic Turning Point
Old Router New Router
to tn
tf
UTP=Unstable Time Period
STP=Silence Time Period
HTP=Handoff Time Period
HTP may be a good reference to the buffer level.
New stream
Old stream
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Timing Analysis UTP = max(0,Tlast_of_old- Tfirst_of_new)
= max(0, (to - tn) + qo + tf)
STP = min (Tfirst_of_old, Tfirst_of_new)
HTP = UTP +STP + |Tfirst_of_new – Tfirst_of_old|
Queueing Model
K-JIST Transient Time Analysis:K-JIST Transient Time Analysis: Handoff Handoff Period Time (HTP)Period Time (HTP)
qo
(a) Before handoff
(b) After handoff
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
K-JIST Transient Time Analysis: K-JIST Transient Time Analysis: AAnalysis Resultnalysis Result
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Seamless Streaming Aid by Adaptive Playout?
Role of Clients for Adaptive Playback
AudioVideo
Controller
Retransmission Buffer
Receiver BufferDemuxAD Buffer
VD Buffer
AD
VD
Composition MemoryCompositor
Controller
RTP Channel
RTCP Channel
Feedback
Playback
Scheduler
sp
p
p
sb
MIN MIW MAW MAX
Playback Speed
MIN
MAW
p
sp
p
Merits of Client Adaptation
•Can prevent excessive buffer overflow and underflow
•Can reduce discontinuous playback time
•Can maintain small buffer size.
Buffer level
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Network Feedback
Layered Video Encoding Network Adaptation
LayeredEncoding
R-DAnalysisVideo
pre-processing
SourceRate/Error
Resilience
DeliveryWith
NetworkAdaptation
(Rate, Delay,Loss)
NetworkMonitoring
&FeedbackHandling
InformBandwidth
&App.
feedback
QualityControl
FrameComplexity
Video packetStream
Receiver
Receiveradaptation
End-to-End Feedback
Application Feedback
WirelessNetwork
R-D /Corruption
Model
LayeredRPI
Seamless Streaming : Network Adaptation
Robust/Scalable Media for Wireless Media Delivery and Associated Network Adaptation
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
End-to-end QoS Challenge: Wireless End-to-end QoS Challenge: Wireless LANLAN
Relative Differentiated Services and the Proportional Differentiation Model for IEEE 802.11x WLAN
o Service Differentiation : Upstream
Different IFS (Inter Frame Space) Different Back-off Time Different Frame Size
o Service Differentiation : Downstream
Scheduler in Access Point
– qi : Performance measure for class I– c1 < c2 <….< cN : Quality differentiation parameter
Proportional Delay Differentiation Model Proportional Loss Differentiation Model
APj
i
j
i
c
c
q
q
q1
q2
q3
Scheduler
Classifier
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Collaboration Issue for H-G Joint ProjectsCollaboration Issue for H-G Joint Projects
With H. Morioka@ISIT, Fukuoka, Japan How to combine ISIT’s Fast and Secure Handoff Idea ? (Short-ter
m)o With implementation: Deploy ISIT’s handoff at K-JIST and test its p
erformance for seamless streaming
o MISP (mobile Internet Service Protocol) for Authentication, Address Assignment, and Encryption
Joint Project (Long-term)o Build ubiquitous computing environment ? Mobile Node
WLAN Interfaces
Access Point
Communication Scanning & Connection
Access Point
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: ISIT: Handover with 2 Interfaces (1)Handover with 2 Interfaces (1)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Handover with 2 Interfaces (2)ISIT: Handover with 2 Interfaces (2)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Handover with 2 Interfaces (3)ISIT: Handover with 2 Interfaces (3)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
ISIT: Fukuoka Mobile Broadband (FMBB)ISIT: Fukuoka Mobile Broadband (FMBB)
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Thank You!Thank You!
NETWORKed MEDIA LAB.
DEPT. OF INFO. & COMM., K-JIST
Ohmuta
Kurume
NogataIizukaTagawa
Fukuoka
Kitakyushu
Kitakyushu CityHigh-speed LAN
QGPOP (ISIT)
To Tokyo
Research Network in KoreaKorean Partner CarrierKJCN Japanese Partner CarriersFukuoka Gigabit HighwayGENKAI ProjectQGPOP part
KJCN
Korea
Busan
KISTI
KIST
GENKAI/Hyeonhae Project
APANInternet2
GbE
2.4Gbps150Mbps
ShimonosekiUniv. of Fishery
FGH AP
Japan
Submarine cable landing station
Network equipment (Routers etc.)
250 km
Repeaterless
KJISTGwangju