Media File Formats for the Media File Formats for the BroadcasterBroadcaster
UnravelingUnraveling the the mysterymystery
Tom Beckers Matthias De Geyter
Wim Ermens Luk Overmeire
VRT Medialab - 16/10/2008
VRT-medialab: onderzoek en innovatie2
Agenda
Part One - Luk Overmeire & Matthias De Geyter
Introduction
Image and advanced compression formats for HD
File formats for HD production: MXF and AAF
--- pause ---
Part Two - Tom Beckers & Wim Ermens
Visual and technical quality control for HD
Demos
VRT-medialab: onderzoek en innovatie3
DistributionProduction infrastructure
Information management
Infrastructure(storage and network)
VRT Medialab4 research domains
IngestMedia
Asset Mgnt
Editing
Playout
Mastering
Manufacturing
EngineeringDistribution
Automation
Product
Development
Information front-end
Research Archiving
broadcaster
informationessence
customer
Internet
Mobile
Wireless andwired networks
Satellite
Personalization
User interaction
User generatedcontent
Social platforms
VRT-medialab: onderzoek en innovatie4
Production infrastructure
Infrastructure(storage and network)
VRT Medialab4 research domains
IngestMedia
Asset Mgnt
Editing
Playout
Mastering
Information management
Manufacturing
EngineeringDistribution
Automation
Product
Development
Research Archiving
Distribution
Internet
Mobile
Wireless andwired networks
Satellite
Information front-end
Personalization
User interaction
User generatedcontent
Social platforms
VRT-medialab: onderzoek en innovatie5
Context: tape-based mediaproduction
VRT-medialab: onderzoek en innovatie6
essence
information
Context: file-based mediaproduction with central warehouse
essence
information
VRT-medialab: onderzoek en innovatie7
Central storage
INGEST
Media
Asset Mgmt
IP network
Central
Storage
Draft Processing
Concurrent
engineering
Browse Clients Processing
during
ingest/transfer
POSTProduction
Craft
editors
Subtitling
Archiving
Analysis
Annotation
Follow-upcontinuity Editor-in-Chief
Post-sonorisation
PLAY-OUT
File-based media production
VRT-medialab: onderzoek en innovatie8
Media production formats
image formats
! resolution, frame rate, raster
compression formats
! high & low resolution
file/wrapper formats
essence
information
=essence
information
+
VRT-medialab: onderzoek en innovatie9
Typical broadcast chain
production
contribution
archive
distributionBlu-ray DVD
linear
Web
I
N
G
E
S
T
P
L
A
Y
O
U
T
central
warehous
e
post
production
Studio acquisition
File-based camera
VRT-medialab: onderzoek en innovatie10
Typical broadcast chainFormats for Standard Definition
production
contribution
archive
distributionDVD
linear
Web
I
N
G
E
S
T
P
L
A
Y
O
U
T
central
warehous
e
post
production
Studio acquisition
File-based camera
MPEG-2
long GOP
4:2:0
8-10 Mbps
MPEG-2
long GOP
4:2:2
10-25 Mpbs
DV25
IMX-30
IMX-50
576 lines, interlaced
25 frames/sec
MPEG-1
VC-1
MXF OP1a
DV25
IMX-30
IMX-50
MXF/AAFMXFH.263/VC-1
MPEG-2
long GOP
VRT-medialab: onderzoek en innovatie11
Typical broadcast chainFormats for High Definition
production
contribution
archive
distributionBlu-ray DVD
linear
Web
I
N
G
E
S
T
P
L
A
Y
O
U
T
central
warehous
e
post
production
Studio acquisition
File-based camera
H.264 4:2:0
bit rate ?MPEG-2 ->H.264
4:2:0, 4:2:2
bit rate ?
XDCAM HD,
AVC-Intra,
DNxHD,
ProRes,
JPEG2000
720p50, 1080i25,
1080p25, 1080p50
H.264 (1),
H.264 (2),
…
MXF OP1a
MXF/AAF,
QuicktimeMXF
XDCAM HD,
AVC-Intra,
DNxHD,
ProRes,
JPEG2000
uncompressed,
lossless compressed ?
H.264/VC-1
H.264/VC-1
long GOP
VRT-medialab: onderzoek en innovatie12
Proliferation of media formats
HD image formats
! 720p50,1080i25, 1080p25
=> 1080p50
HD compression formats
! legacy versus new formats
! contribution, camera, production, archive, distribution
proxy formats (low resolution)
wrapper formats
! MXF flavors, Quicktime
VRT-medialab: onderzoek en innovatie13
The HDTV format jungle
Physikalischer Transport
IT-basiert: File Format
Kompressionsformat
Abtastformat
Physikalischer Transport
IT-basiert: File Format
Kompressionsformat
Abtastformat
Physikalischer Transport
IT-basiert: File Format
Kompressionsformat
Abtastformat
Physikalischer Transport
IT-basiert: File Format
Kompressionsformat
Abtastformat
Physical Transport(HD-SDI, IP, Tape, ..)
File Formats
(MXF, QT, ..)
Compression formats(MPEG-2, H.264, JPEG2000, ..)
Sampling formats(1080i/25, 720p/50, 1920/1440 pixels,
4:2:0, 4:4:4, ... )
Production
(more granular variants for news, features, etc.)
Distribution
Contribution („live“ and/or „File “)
Presentation (Displays)
Lossy
conversions
HDTV increases
variety of formats
HDTV increases
variety of options
„A format“
Further formats (e.g. consumer)
HDTV increases need
for efficient wrapping
VRT-medialab: onderzoek en innovatie14
Production areas
high-end production
! landmarks, drama
mainstream production
! soap series, documentaries, shows
News and sports
video journalism
! consumer cameras
" single or multiple scanning formats ?
" single or multiple compression formats ?
" 720p/50 a/o 1080i/25
" 4:2:2 a/o further
subsampling
" 10 Bit a/o 8 Bit
" I-frame only a/o long GOP
" Headroom a/o low data rate
" … a/o …
Mainstream HDTV platform
Image formats for HD(and beyond)
VRT-medialab: onderzoek en innovatie16
EBU
Tech 3299
HDTV System 1-4
Today’s and future image formats
Future
Today
System 2 – 1080p25
resolution 1920 x 1080
25 frames/sec
progressive
3 Gbit/s
1,5 Gbit/s
System 3 – 1080i25
resolution 1920 x 1080
50 fields/sec
interlaced
System 1 – 720p50
resolution 1280 x 720
50 frames/sec
progressive
System 4 – 1080p50
resolution 1920 x 1080
50 frames/sec
progressive
Super Hi-Vision (U-HDTV)
resolution 7680 x 4320
60 frames/sec, progressive
audio: 22.2 channels
Far future 24 Gbit/s
?
VRT-medialab: onderzoek en innovatie17
EBU – Tech 3299
2,0736104:2:25010801920System 41080p/50
1,0368104:2:22510801920System 31080p/25
1,0368104:2:22510801920System 21080i/25
0,9216104:2:2507201280System 1720p/50
Net image BitRate [Gbit/s]
Sub-sampling /Quantisation [Bit]
Framerate
Activelines
Horizontalsamples
HDTVSystems
EBU 3299
Interface BitRate [Gbit/s]
1,5
1,5
1,5
3,0
0,270,207104:2:225576720SDTV
576i/25
VRT-medialab: onderzoek en innovatie18
The resolution revolution
Maximal viewing distance:
SD: d < 5.h
HD: d < 3.h
U-HDTV: d < 0.75.h
d = viewing distance
h = display height
now
+ 5y
+ 15y?
VRT-medialab: onderzoek en innovatie19
Good HDTV – More than anumbers game
1080 (i) > 720 (p) => “More is better” ?
VRT-medialab: onderzoek en innovatie20
How can smaller be better ? (1)
interlace factor
! interlaced scanning: 50 fields/sec (1 field = ! frame)
! 540 lines < dynamic vertical resolution < 1080 lines
Studies (Bell Laboratories, NHK): interlace factor ! 0.6 => 648 lines < 720 lines
VRT-medialab: onderzoek en innovatie21
How can smaller be better ? (2)
interlace factor
interlace artefacts (motion rendition)
! e.g. serrated edges
VRT-medialab: onderzoek en innovatie22
How can smaller be better ? (3)
interlace factor
interlace artefacts
resolution capacity of human eye (" 1 arc minute)
Required horizontal sampling for different screen sizes, viewed at 2.7m
720x576
VRT-medialab: onderzoek en innovatie23
How can smaller be better?
interlace factor
interlace artefacts
resolution capacity of human eye
compression artefacts
! better compression performance for progressive
! compression of interlaced media generally involves de-interlacing
VRT-medialab: onderzoek en innovatie24
Progressive versus interlaced
far better suited for moving pictures (e.g.sports)
easier to encode
plasma, LCD are progressive displays
=> end-to-end progressive workflow is possible
same resolution visibility from “viewingdistance”
rescaling is easier than deinterlacing
the future is progressive => evolution to1080p50 is more straightforward
legacy format
! as of today, generally bettersupported throughout the workflow
– camera, set-top-box, postproduction, central MAM, …
! existing program material
! current offering is predominantly
1080i25 content
compatible with 1080p25 through 1080psf
Pro
Contra
full support in end-to-end workflows to becarefully tested
! camera (quality), central MAM, postproduction, Dolby E surround sound
! time labeling ! (editing)
! distribution (Telenet, Belgacom, …)
! contribution (productiehuizen)
higher complexity
! down conversion requires de- and re-interlacing
! de-interlacing is very hard to do well
legacy format - “wet van de remmende voorsprong”
less suited for slow motion and chroma keying
VRT-medialab: onderzoek en innovatie25
720p
Different scenarios for imageformats
1080i 1080i
Acquisition
1080ito
1080p
Production
1080i
Distribution
1080i 1080p
Full HDDisplay
1080i 1080i 1080i 720p1080i
to720p
720pto
1080p
1080p
Deinterlacingconverter
Rescalingconverter
Scenario 1
Scenario 2
1080i1080i
to720p
720p 720p720p
to1080p
1080pScenario 3
Scenario 4 720p 720p 720p 720p720p
to1080p
1080p
Distribution
format
Production
format
Best quality
What is effect
of conversion
on quality?
Best supported
VRT-medialab: onderzoek en innovatie26
Broadcast production chain –Points of attention for 720p50 workflows
Studio camera
Cross-convertorSet-top-box
Real-time infrastructure
File-based production environment
Cross-convertor
MXF
Central MAM
Post productionSurround sound
p50 timing
End-to-end workflow should be carefully designed !
1
Today’s content is mostly 1080i
2
Motion compensation algorithm
5
Support (legacy systems)
3
Light sensitivity, S/N ratio
4 Full tested support (essence, TC, …)
7 Dolby E compatibility with p50
6 TC support (legacy formats)
8 EBU Joint Taskforce on
Timing & Synchronisation
9EICTA label - HDTV
VRT-medialab: onderzoek en innovatie27
EBU/SMPTE Joint Taskforce onTime Labelling and Synchronization
twofold goal
! new time labelling method allowing higher frame rates (50 fpsand higher)
! new synchronization mechanism
! current standards (SMPTE 12M, colour black) are 30 years old
=> workable but limited
status
! Request For Technology (RFT) – February 2008
– responses from Sony, Harris, TGV, …
! draft Request For Standardisation (RFS)
– time-related labeling
VRT-medialab: onderzoek en innovatie28
Scenarios for 1080p25
1080p25 1080p25
Acquisition
1080psfto
1080p
Production
1080p25
Distribution
1080psf(1080i)
1080p
Full HDDisplay
1080p25 720p1080p25
to720p
720pto
1080p
1080p
Rescalingconverterwith loss of resolution
Rescalingconverter
Scenario 1
Scenario 2 1080p25 1080p25
psf = progressive with segmented frames
! frames are segmented into two interlaced fields with equal timeindex => compatible with 1080i distribution
! easy deinterlacer (simple “weave”)
1080p25to
1080psf
Simplereordening
VRT-medialab: onderzoek en innovatie29
What about 1080p50?
no one already broadcasts in 1080p50
! but some have plans …
1080p50 in production
! not around the corner, but plausible within 3-5 years
! very good master for both 720p50 and 1080i25
! better quality (less artifacts due to smaller block sizes andhigher coding efficiency)
1080p50 in distribution
! commercial factors will likely be the driving factor
Compressionformats for HD
VRT-medialab: onderzoek en innovatie31
Basics of video coding
What is video compression?
! Video compression is a technique that reduces the bit rate torepresent video images by exploiting signal redundancies whilepreserving the original quality as much as possible
basic types of redundancies
! statistical redundancies
– spatial and temporal interpixel redundancies
– coding redundancy
! psycho-visual redundancies
– human visual system is less sensitive to higher frequenciesand color
VRT-medialab: onderzoek en innovatie32
Common techniques for videocompression
source
encoderre-
quantizationentropy
encoding
input
videocompressed
video
subsampling transform
horizontal
subsampling
color
subsampling
DCT
wavelet
prediction
Intra-frame
Interframe
lossless
vs lossy
rate control
CBR/VBR
VRT-medialab: onderzoek en innovatie33
Color subsampling
source
encoderquantization
entropy
encoding
input
videocompressed
video
subsampling transform
horizontal
subsampling
color
subsampling
DCT
wavelet
prediction
Intra-frame
Interframe
lossless
vs lossy
rate control
CBR/VBR
color
subsampling
VRT-medialab: onderzoek en innovatie34
Typical compression ratios
3/4color subsampling 4:2:0
4/5Quantization = 8 bits
3/4horizontal subsampling
50-200 Mbps
35-50 Mbps
1/5-1/20
1/20-1/30
Intra-frame compression vs
Interframe compression
500 Mbps1/2Lossless encoding
1,5 Gbps
(net " 1 Gbps)
1HD-SDI (4:2:2 subsampling)
Video bit rateFactorCoding technique
Reference = HD-SDI 720p50/1080i25 (10 bit)
VRT-medialab: onderzoek en innovatie35
Mainstream HD productionplatform – codec dilemmas
horizontal subsampling?
color subsampling: 4:2:2 or 4:2:0?
8 or 10 bit quantization?
intra-frame or interframe coding?
scalable or single-layer compression?
single or multiple compression formats?
required bit rate?
720p or 1080i?
VRT-medialab: onderzoek en innovatie36
Legacy HD production formats
Inter4:2:0; 8 bit
Horizontal
subsampling
20/25HDV
Inter4:2:0; 8 bit
Horizontal
subsampling
35XDCAM HD 35
Intra4:2:2; 8 bit
Horizontal
subsampling
100DVCProHD
Intra4:2:2; 8 bit
Horizontal
subsampling
140 (*)HDCAM
Intra4:2:2 or 4:4:4
10 bit
440HDCAM SR
Intra/interframe
coding
Subsampling
Quantization
Bit rate (Mbps)Compression
algorithm
(*): net video rate = 117Mbps
VRT-medialab: onderzoek en innovatie37
Advanced HD compression formats formainstream production
XDCAM HD
MPEG-2 Long-GOP
4:2:2, 8 Bit, data rate 50 Mbps, VBR
P2 HD
AVC Intra
4:2:2/4:2:0, 10/8 Bit, data rate 100 / 50 Mbps, CBR
EditcamHD
DNxHD
4:2:2, 8 Bit, Data rate 120 Mbit/s (185 Mbps), CBR
Infinity
JPEG2000
4:2:2, 10 Bit, Data rate 100 / 75 / 50
Mbit/s
Camera formats (optimized for storage and power
consumption)
VRT-medialab: onderzoek en innovatie38
Advanced HD compression formats formainstream production
post-production formats (optimized for editing)
185 / 120Net video data rate[Mbit/s]
4:2:2Colour sub-sampling
8 / 10 BitQuantisation [Bit]
SMPTE VC-3Standard
185 / 120Net video data rate[Mbit/s]
4:2:2Colour sub-sampling
8 / 10 BitQuantisation [Bit]
---Standard
DNxHD ProRes"
“intermediate codecs”
VRT-medialab: onderzoek en innovatie39
MPEG-2 long GOP
aka XDCAM HD422
advantages
! proven and optimized codec
! same bit rate as for Standard Definition (50Mbps)
! in general: good quality for first generation
! early support for end-to-end broadcast chain
disadvantages
! interframe is less suited for editing purposes
– smart rendering techniques necessary
– fade-to-black, dissolve effects
– artefacts affect multiple frames
! quality loss in multi-generation scenarios
successor = H.264 long GOP?
VRT-medialab: onderzoek en innovatie40
H.264 - AVC Intra
H.264 = MPEG-4 AVC = MPEG-4 Part 10
! joint project of ITU and MPEG
addition of High Profiles for HD resolution
! FRExt (Fidelity Range Extension)
– higher bit depths (10), 4:2:2/4:4:4 colour formats
! High 10 Intra (50 Mbps) and High 4:2:2 Intra (100 Mbps)
advanced compression tools
! Intra prediction: use of correlation between adjacent pixels
! entropy encoding improvement
– CABAC: Context Adaptive Binary Arithmetic Code
point of attention
! higher computational complexity
VRT-medialab: onderzoek en innovatie41
JPEG 2000
available bit rates: 50/75/100 Mbps
advantages
! scalable codec: no need for proxy transcoding
! wavelet-based coding: no blocking artefacts
! industry standard for Digital Cinema
! low latency
! license free
disadvantages
! not widely supported in end-to-end broadcast chain at thismoment
VRT-medialab: onderzoek en innovatie42
Dirac codec family
developed by BBC R&D (2000-now)
! license-free, open source/well documented
wavelet-based codec
wide range of applications
! Dirac (long GOP): low bit rate distribution (mobile, Web)
! Dirac Pro (intra-frame): low latency, high-quality production(HD, DC, Super Hi-Vision)
– SMPTE VC-2
– transport mapping to SDI (720p50/1080i25) and HD-SDI(1080p50)
! high flexibility: lossy/lossless, variable bit depths and coloursample schemes, …
not yet supported, less mature than other HD codecs
VRT-medialab: onderzoek en innovatie43
Scalable video coding (SVC)
extension of H.264
! recently standardized (2007)
! base layer (H.264) + enhancement layers
scalability options
! spatial, temporal, SNR/quality, combined
possible applications
! alternative for simulcast: SD/HD
! HD TV distribution for 720p@50fps or 1080i@25fps with1080p@50fps enhancement
! broadcast production for different resolutions
targeted profiles for broadcasting
VRT-medialab: onderzoek en innovatie44
Scalable video coding (SVC)
extension of H.264
! recently standardized (2007)
! base layer (H.264) + enhancement layers
scalability options
! spatial, temporal, SNR/quality, combined
possible applications
! alternative for simulcast: SD/HD
! HD TV distribution for 720p@50fps or 1080i@25fps with1080p@50fps enhancement
! broadcast production for different resolutions
targeted profiles for broadcasting
VRT-medialab: onderzoek en innovatie45
Choice of HD compressionformat
key parameters
! quality (desired, floor level)
! storage and bandwidth requirements
! workflow
– recording medium
– editing requirements: layers, editing easiness, …
– support by different systems in HD production architecture
– end-to-end performance (transcoding, rendering, …)
! when?
VRT-medialab: onderzoek en innovatie46
Mainstream HD productionplatform – codec dilemmas
horizontal subsampling?
color subsampling: 4:2:2 or 4:2:0?
8 or 10 bit quantization?
intra-frame or interframe coding?
scalable or single-layer compression?
single or multiple compression formats?
required bit rate?
720p or 1080i?
no
4:2:2
8/10
50/100 Mbps