A Web Services Based Streaming Gateway for Heterogeneous A/V ...

A Web Services Based Streaming Gateway for Heterogeneous A/V

Collaboration

Hasan Bulut

Computer Science DepartmentIndiana University

Motivation Videoconferencing systems provide a framework

to send/receive audio and video streams. Streaming is a widely used media delivery

technology across Internet to receive Media-On-Demand. It enables users to receive multimedia content and provides some control functionality over the media.

To enable streaming clients to join the real-time videoconferencing sessions and receive multimedia content generated by other clients in the system.

Videoconferencing Systems

Major videoconferencing systems are: H323

H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment.

SIPThe Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences

Access Grid Enhanced Mbone A/V tools ( VIC, RAT ).

Internet 2 network ( Multicast support )

Videoconferencing vs Streaming

A streaming client connects to a streaming server, primarily using RTSP, to establish a session and receive the stream.

Streaming is primarily used for Media-On-Demand. It is also possible to make a live streaming broadcast.

Streaming media aims to improve the quality of service of the stream, by using various streaming codec to improve stream quality and a buffering mechanism to reduce the jitter.

While the delay in conferencing is around 200 msec, the delay in streaming buffer is around 2 – 15 sec.

In this paper we will explain the components of

XGSP Streaming Gateway, their functionality and messages added to the framework to achieve integration. Other components will be briefly explained.

GlobalMMCS, which implements XGSP, uses Web Services technology to integrate heterogeneous conferencing systems. Streaming Gateway has been recently added to this system and it is used internally by other components in the system.

XGSP Collaboration Framework To integrate heterogeneous systems into one

collaboration system, we need to reach the following goals:

Different kinds of application endpoints should join leave in the same collaboration session.

A common A/V signaling protocol has to be designed to support interactions between different A/V collaboration endpoints.

A common user interface should be present for all the collaboration participants using different A/V and data application endpoints.

Different A/V endpoints should collaborate in the same collaboration session.

Global Multimedia Conferencing Systems

(GlobalMMCS)The first prototype of this system includes: A XGSP media server

provides the services of bridging multicast and unicast, video-switching, video-mixing and audio-mixing to H.323, SIP as well as AG endpoints.

H.323, SIP gateways and Real Servers for A/V clients XGSP A/V Session Server

manages real-time A/V sessions, receives messages from gateways and the web server, and performs appropriate actions on the media server.

The web server provides an easy-to-use web interface for users to

join multimedia sessions and for administrators to perform administrative tasks.

GlobalMMCS Architecture

RealStreaming Clients

SIP Gateway

Media Servers;Audio Mixers,Video Mixers,Image Graber

SIP Clients

XGSP Session Server

XGSP Web Server

NaradaBrokeringPublish/Subscribe

Messaging

H.323 Gateway

Access Grid Multicast

Streaming Gateway

Helix ServerH.323

Clients

Two-way RTP-link/linksOne-way RTP-link / linksPushed streamsStreamingCommunication link / links

XGSP Streaming Gateway(XGSP–SG)

Due to the requirements and underlying technology of streaming clients, XGSP–SG demonstrates different functionalities compared to other gateways:

Only uses XGSP message formats, while other gateways only transforms XGSP signal to another form and vice versa.

Performs stream format conversion so that streaming clients can play.

Provides a mechanism to synchronize streams Streaming gateway allows streaming clients

only to receive audio and/or video streams.

XGSP–SG Components and Design

Stream EngineS

tream

C

onvers

ion

Han

dle

rSMIL File Generator

Helix Streaming Server

Stream Engine

Communication linkPushed streamsProcess generation

Streaming Gateway components

XGSP–SG Components I Stream Conversion Handler

performs the communication between Session Server and Streaming Gateway.

maintains an internal database for the streams. Database is updated when the streaming jobs are started or deleted.

initiates/stops Stream Engines for the requested stream

XGSP–SG Components II Stream Engine

converts received audio or video streams into a specified RealStreaming format and pushes the output stream to Helix Streaming Server

composed of two parts, JMF RTP Handler and HXTA Wrapper

JMF RTP Handler: transform the received packets into raw audio or video format

HXTA Wrapper: makes use of HXTA engine to encode streams into RealMedia format

XGSP–SG Components III SMIL File Generator

dynamically constructs a SMIL file using the received audio and video stream IDs

deletes SMIL file if the stream is no longer in the session

SMIL files enables clients to receive multiple streams and synchronize them

Each session in a XGSP session contains only one audio stream, which is the mixed of all audio streams received from clients.

XGSP–SG Messages I InitializeRealGateway - deletes all current

jobs in the session specified JoinStream - starts a job for the stream

specified. JoinStreamReply - reply message for

JoinStream message. OK indicates the job is successfully started, otherwise returns FAIL.

LeaveStream - stops the stream specified. LeaveStreamReply - reply message for

LeaveStream. FAIL indicates that an error occurred during the stop operation, otherwise returns OK.

XGSP–SG Messages II RealStreamEvent - It has two modes,

NewRealStream to indicate a new stream in the session and ByeRealStream to indicate the stream is no longer in the session

RealStreams - requests the list of the RealStreams available in the session

RealStreamsReply - reply message for RealStreams. Includes a list of the RealStreams in the session.

Services added to Session Server

RS Join Service - handles JoinStream messages. If stream job is successfully started, sends RealStreamEvent message (mode = NewRealStream).

RS Leave Service - handles LeaveStream messages. Sends RealStreamEvent (mode = ByeRealStream).

RS List Service - handles RealStreams messages. Replies back with RealStreamsReply listing all the available RealStreams in the session.

RS Gateway Service - handles InitializeRealGateway messages. Sends RealStreamEvent with ByeRealStream mode for each RealStream in that session.

Streaming Interface I Two types of interfaces are developed for

Streaming Gateway. Streaming Admin Interface: provides

administrative features such as starting and stopping streaming jobs.

Streaming Client Interfaces: enables clients to plays streams from an interface

Streaming Admin interface also includes Streaming Client interface functionality to enable administrator with client capabilities.

Streaming Interface II A snapshot from Streaming Admin interface

Streaming Interface III A snapshot from Streaming Client interface

Streaming Interface IV A snapshot of the stream played in RealOne

Player window

Performance Discussion for Streaming Gateway I

Number of

Stream

Frame rates of streams

involved

CPU UsageRange

MemoryUsage

1 23 fps %10 - %25 29 MB

2 23 fps, 25 fps %22- %40 34 MB

3 23 fps, 25 fps, 26 fps

%50 - %75 43 MB

4 23 fps, 25 fps, 26 fps, 16 fps

%65- %95 53 MB

OS Windows XP

CPU Intel® Pentium® 4 CPU 2.26 GHz

Memory 512 MB

JVM version 1.4.2_02

Approximate CPU and memory usage with respect to number of streams

Streaming Gateway machine specifications

Performance Discussion for Streaming Gateway II

In a session (or classroom) usually there is one presenter. Sometimes other clients in the session may be active too.

In a session most of the clients are only listeners.

In such a session providing 3 video streams and a mixed audio stream ( mixed of all audio streams received from clients in the session) is sufficient.

Conclusion and Future Work XGSP framework and, Global-MMCS enable

multiple communities to collaborate with each other.

Streaming Gateway enriches this heterogeneous system by integrating streaming media world into Global-MMCS.

Streaming Gateway is part of Global-MMCS and uses XGSP in order to achieve this integration.

In order to increase the number of streams converted, we also would like to build a streaming job scheduler, which will schedule and coordinate streaming jobs in a distributed environment.

Thank you!