02-Video Processing for Mobile Communication(구준모)umls.kaist.ac.kr/professor/ftp/4th_workshop/02-Video... · 2011-07-19 · ¤Moving picture + speech = 15sec ... Inverse Discrete

Video processing for mobile

communication

2002.02.05

구준모

엠큐브웍스

CRLCRL제제44회회 추모워크샵추모워크샵

Multimedia ServicesMultimedia Servicesin the Next Generation Networksin the Next Generation Networks

R/Evolution of Mobile Network

n 2G¨ IS95-A, GSM, PDC : ~10Kbps

n 2.5G¨ IS-2000 1x : major cities in Korea, peak 144Kbps¨ GPRS : Europe/US, 40~50Kbps

n 3G¨ IS-2000 EVDO : from ’02, peak 2Mbps¨WCDMA : from ’03, peak 2Mbps

n 3.5G¨ Public WLAN : from ’02, peak 11Mbps



Market Drivers

n Internet¨ A part of daily lives¨ Data/Multimedia service¨ Overlap of Internet users and Mobile users

n Carrier¨ Saturation of mobile voice service¨ Subscribers and fee¨ Competition

n Government¨ Industry based on new technology¨ Strategy for export



Evolution of Service

n Present¨ Circuit : voice, SMS, VMS, IN based¨ Packet : WAP based low rate data

n 벨소리 다운로드, 그림친구, 게임 : 50% of mobile Internet revenue

n Future¨ Circuit : voice, UMS, IN based¨ Packet : Internet based high rate data

n MMSn Entertainmentn Financial servicen Commerce



A Goal - Mobile Multimedia

n Multimedia In Your Hand¨ Carrier

n Heavy data traffic : up to 10 times

¨ Vendorn New demands for system and terminal

¨ Contents Providern Double the market

¨ Usern Some critical service : Map viewn Enhancing communication : Tour informationn New service : MP3 download, Video chatting

¨Multimedia a new source of revenue



Challenges

n Is it possible?¨Multimedia requires

n Lower error rate as compared to voice¨ 10e-7 vs 10e-3

n High computing power¨ Video encoding

¨ High power consumption

n Complex protocol¨ Signaling such as H.323, SIP

¨ Transport such as RTP



Challenges -Radio

n Radio Channel¨ 10e-3 for voice and 5% FER for packet

¨ Error shows burst pattern

¨ Dynamic variation of throughputn IS-2000 1x : 9.6Kbps ~ 144Kbps

n Traffic dependent

n Location dependent

n Entry control

¨Mobilityn Handover : SCH -> FCH -> SCH

¨ Time Varying Channel in Throughput and Error



Challenges - Network

n Packet Network With Non-IP Node

Switch

A1 A2

Call Control,

Mobility

Management.

MSC

A3(user traffic)

IWF

A5

Target BS

A7 (signaling)

A3(signaling)SDU

Function

BTSBTS

BSC BSC

PCF PDSN

A10 (user traffic)

A11 (signaling)

A8 (user traffic)

A9 (signaling)

Source BS

(This interface is not included in this specification.)

Ater Reference Point

Aquater Reference Point



Challenges - Network

n Heterogeneous Protocol Stack

PHYS

LinKLayer

PHYS

PDSN

PHYS

Link Layer

Airlink

RelayFunction

GRE

BS/PCF

IP

GRE

IP

AdaptationLayer

AdaptationLayer

Link Layer

LAC

MAC

IPIP

UDP

MIP

PPP/DL

MobileStation

MIP

IP

PPP/DL

Airlink

LAC

UDP

MAC

PHYS

Link

Layer

IP

UDP

AppApp

Server



Challenges - Terminal

n Voice Terminal¨ Limited computing power

n MSM-5000 : 5~8Mipsn Chips for multimedia processing : ARM, DSP

¨ Limited power supply and peripherals¨ Various output device : BW, 4Gray, 2” color

display

n PDA¨ Enhanced computing power : 206MHz, SA1110¨ Large screen with plugging peripherals¨ Limited power supply due to display



Challenges – User Behavior

n Video Watching¨ Fixed location

¨ Free(TV) or very cheap per unit time(movie)

¨ No experience of face-to-face communication

n Recent changes¨ Internet

¨ Pulling video from Internet

¨ Video chatting

¨ Free or flat rate



Challenges - Fee

n Expensive For Multimedia Service¨ Korea

n 2.5 won per packet(=512Byte=4Kbit)n 30sec. 32Kbps multimedia = 2.5*30*32K/4K = 600 wonn 30sec. voice = 20*30/10= 60 won

¨ Japan(DoCoMo i-motion) n 1 packet = 0.2 yenn Almost same as Korean Provision plan

¨ Moving picture + speech = 15sec¨ Still picture + speech = 30sec¨ Speech = 100sec

n Flat rate plan for PHS m-stage



A solution - Multimedia Processing

n Algorithms For Mobile Environment¨ High compression

n Critical for the efficient use of spectrumn Different Quality at the same rate

¨ Low complexityn Reducing computational burden at terminaln SW codec reducing terminal cost

¨ Error robustnessn Error concealmentn Error resilience

¨ Small foot printn Memory size affects the terminal cost and size



Audio/Speech Coding

n Speech Codecs¨Mobile environment is considered

¨ 3GPP : AMR, WB-AMR

¨ 3GPP2 : EVRC, SMV

¨ ITU-T : G.723.1

n Audio Codecs¨ Required bandwidth : MP3 > AAC >WMA

¨Mobile environment is not considered



Video Signal Processing

n Video Signal¨ 70% of information comes from eyes

¨ Very large quantity of data

¨ Highly correlated between frames

¨ Error sensitive

n Frame data¨ Picture = sequence of frames

¨ Frame = set of pixels(QCIF:176x144)

R G B Y U V



Video Signal Processingn Compression¨ Use of correlation between frames(I,P,B)¨ Processing in transformation domain(DCT)¨ Motion Estimation ¨ Quantization and Variable Length Coding

capturecapture DCTDCT QuantizationQuantization VLC/MUXVLC/MUX

I D P- =

PredictionMemory

PredictionMemory

LoopFilter

LoopFilter

DequantizerDequantizer

IDCTIDCT

IDCT: Inverse Discrete Cosine TransformVLC: Variable Length Coding

MotionEstimation

MotionEstimation



Video Signal Processing



Processing For Mobile Video

n Spectrum Efficiency ¨ Codec for synthetic image

n Very low rate video coding

n SIS by NEOMTEL : parametric coding

n 3G Image coding

¨ Codec for real imagen Usually standard codecs show better performance than

non-standard codec

n JPEG-2000

n H.263

n MPEG4




n Complexity ¨ Low complexity codecs for mobile handset

n Wavelet codec by TMIn Light codec by Togabin Runs at Qualcomm MSM5xxxn Proprietary codecs n Low compression performance or low quality

¨ Complexity enhancement of standard codecsn Without post filter : MPEG4

¨ Encodern Always requires special purpose hardware in the present

handset




n Error Robustness ¨ Error concealment

n Recovering image from the corrupted imagen Image copy, data partitioning(important bit -> front),

Adaptive intra refresh(motion part -> intra refresh)

¨ Error resiliencen Making image robust at transmission errorn Resync marker, Reversible VLC, Header Extension Code

¨ Packet level errorn Mobile video using IP protocoln Packet level error rather than bit leveln Different error control algorithm required




n Standard ¨ 3GPPs

n H.263 : mandatoryn H.263+, MPEG4 : optional

¨ ITU-T and ISO/IEC

ITU-TITU-T

ISO/IECISO/IEC

H.261H.261

MPEG1MPEG1

H.262H.262

MPEG2MPEG2

H.263H.263

MPEG4MPEG4

H.26LH.26L

Video phonepx64Kbps, p=1~30

Storage, CDI1.5Mbps

Broadcasting, HDTV, DVD TV @4Mbps

Video phone High performancefull range

Object based coding32K~ fewMbps



Protocols For Mobile Media

n Standard ¨ 3GPP and 3GPP2

n Video telephony¨ Signaling : H.324M(circuit), SIP(packet)

¨ Transport : RTP

n Streaming service ¨ Signaling : RTSP

¨ Transport : RTP

¨ Other bodies n IETF, WMF, ISMA

¨ SIP n New signaling standard for mobile service



Protocols For Mobile Media

n Stack ¨ Based on Legacy Protocol

n H.323, H.324M

¨ Based on Internet Protocoln Signaling

¨ SIP

¨ RTSP

¨ WAP

¨ HTTP

n Transport¨ RTP/UDP/IP for real time application

¨ TCP/IP for non-real time application



Joint Optimization

n Network Measurement

Arrival packet number

Round trip delay

Number of lost packet

Throughput: 30Kbps

Packet size: 400Byte

Interval: 100ms

Packet loss rate: 0.08%



Joint Optimizationn Network Measurement

Arrival packet number

Round trip delay

Number of lost packet

Throughput: 100Kbps

Packet size: 400Byte

Interval: 28ms

Packet loss rate: 36%



Joint Optimization

n Joint Optimization

time(ms) time(ms)

Packet N

umber

Packet N

umber



A Reality - Applications

n VoD ¨ At phone

n TMI : Proprietary

n Togabi : Proprietary

n VaroVision, OnTimeTech: MPEG4

¨ At PDAn McubeWorks

n MicroSoft

n Packet Video



A Reality - Applications

n Video Telephony ¨ At phone

n Samsung, LG, SK Teletech : H.323

¨ At PDAn McubeWorks : SIP

n Multimedia Messaging ¨ Under development

n H.323 based

n WAP, HTTP based



Further Issues

n Fee ¨ Flat Rate or Free

n Wireline → Public WLAN → Cellular

n User Behavior ¨Migration

n Voice → Audio → Photo → Video

n Killer Application ¨Most Important

¨ Different from offline and wireline applications



Further Issues

n Killer Applications of Mobile Multimedia ¨ Readiness

¨ Portability

¨Mobility

¨ Real time, Short clip, Location basedn Multimedia messaging

n Photo mail

n Live



Conclusion

n Mobile Multimedia Service¨ Next service at initial stage

¨ Versatile by integrating with Internet

n Technology Required¨ Different from offline, broadcasting, Internet

¨Media processing(especially video)

¨ Service protocol

¨ Spectrum efficiency

¨ Power consumption

¨ Error robustness

02-Video Processing for Mobile Communication(구준모)umls.kaist.ac.kr/professor/ftp/4th_workshop/02-Video... · 2011-07-19 · ¤Moving picture + speech = 15sec ... Inverse Discrete

Documents