Jim Wilkinson, Co-Chair: File Interchange Working Group, Pro-MPEG

Jim Wilkinson, Sony BPRL

MXF & Metadata Overview

Jim Wilkinson,Co-Chair: File Interchange Working Group,Pro-MPEG Forum

30th January 2004

2 Jim Wilkinson, Sony BPRL

Agenda

Metadata registries

MXF documents & standardization progress

MXF Structural Metadata

DM Plug-in Mechanism


Metadata Registries (in progress)

SMPTE 336MKLV coding of Elements and GroupsThe core of MXF codingProvides for the coding of single elements, groups and labels

SMPTE 335MDefines the structure of the Metadata Dictionary

RP210.7The latest version of the Metadata Dictionary

SMPTE 400MDefines the structure of the Labels Dictionary

RP224.5The latest version of the Labels Dictionary


Metadata Registries (to-do)

Structure document for data typesTo be done real soon!

Data Types RegistryKeenly awaited ….

Structure document for groups, S395MLast balloted with some comments in September 2003No known progress since

Groups RegistrySeen as an early example, but still waiting in the first revision for ballotMuch of the delay due to still developing groups (sets/packs) in MXFAlso issues with the order of elements in sets & packs


Metadata Registries & MXF

Metadata Dictionary

RP210

Enumerations (e.g. SMPTE

Labels, RP224)

Data Types

MXF File Components

MXF Structural Metadata

Model

MXF Descriptive Metadata

Model

Selected Metadata Values

MXF Groups


MXF documents - a modular approach

Part 1EngineeringGuidelines

(informative)

EG41, EG42

Part 2

File Format

SMPTE 377M

Part 3.xOperational

Patternsi.e. constraints on

the format

Part 5.xDescriptive

Metadata plug-ins(normative)

i.e. metadatacollections

Part 4MXF GC(EssenceContainer)(normative)

How to KLV codethe A/V E ssence

Part 4xMapping

documents(normative)

i.e. how to map &index essence in

the container


Primary MXF documents

SMPTE Standards:SMPTE 377M (MXF Format)SMPTE 378M (OP1a)SMPTE 379M (Generic Container, GC)SMPTE 383M GC DV (all DV variants, including DVCAM)SMPTE 385M (Mapping SDTI-CP into the GC)SMPTE 386M GC D-10 (Sony IMX)SMPTE 387M GC D-11 (Sony HDCAM)SMPTE 391/2/3M OP1b/2a/2bSMPTE EG41 (MXF Engineering Guide)


Ongoing SMPTE ballots

Documents still in Tech Committee ballot:SMPTE 380M (Descriptive Metadata Scheme-1)SMPTE 381M (GC MPEG)SMPTE 382M (GC AES3 & BWAV mapping)SMPTE 384M (GC Uncompressed Picture)SMPTE 389M (System Item - Reverse Play)

Documents forwarded to Trial PublicationSMPTE 390M (OP-Atom)SMPTE EG42 (Descriptive Metadata guidelines)


MXF - New and Forthcoming

GC A-law mapping (for XDCAM proxy)Now in a re-ballot

GC System Item General Scheme 1Document to be split into 2 partsTo be re-balloted this month (Feb 2004)Now compatible with the Reverse Play document

Data Stream HandlingIn development - needs 2 documentsTo be newly balloted this month (Feb 2004)

Security FrameworkNeeded for encryption, watermarking ID etcWork on hold, pending clear user requirements


KLV coding throughout MXF

Key-Length-ValueKey: a SMPTE registered 16-byte Universal LabelLength: fixed or variable length of value up to 264 bytesValue: the value as a byte string

Defined in SMPTE 336MEverything in MXF is KLV coded

(Except the optional Run-In)MXF uses:

KLV coding of individual elementsKLV group coding as fixed-length packs and local setsSMPTE Labels - 16-byte ULs whose value is the UL itself


File Structure: as a picture

Footer Partition

Header PartitionPack

HeaderMetadata

EssenceContainer

IndexTable

HeaderPartition

RunIn

(optional) (optional)

HeaderMetadata

EssenceContainer

IndexTable

BodyPartition

(optional)(optional)

Body PartitionPack

HeaderMetadata

IndexTable

FooterPartition

(optional)(optional)

Footer PartitionPack

RandomIndex Pack

(optional)

É NextPartition

É NextPartition

(optional)

(optional)

Header Partition

Body Partition


Metadata in MXF

FileHeader

Generic (Essence ) Container

FileWrapper

1 ContentPackage

Intra-track metadatae.g. FormatTemporalSpatialData streams- subtitling etc.

Inter-track metadata)e.g.FormatTemporalMaterialLabelling and Identification

Header Metadatae.g.Material,CompositionalLabelling and IdentificationCatalogueBusiness (access)

Metadata link

Server Metadatae.g.Labelling and IdentificationCompositionalCataloguePublicationBusiness

SystemItem

PictureItem

SoundItem

DataItem

MetadataTrack

PictureTrack

SoundTrack

DataTrack


How MXF Metadata Works

HeaderPartition

Pack(BodySID)

Essence Container

Preface Identification

Content Storage

Essence Container Data

Material Package Tracks

Sequence

Source Clip

Top-level File Package Tracks

Sequence

Source Clip

Lower-level Source Package

Tracks

Sequence

Source Clip Source Video

Link byBodySID Start Position

Duration

SourcePackage ID Source

Track IDStartPosition Duration

SourcePackage ID

SourceTrack ID

OptionalSource

PackagesHeader Metadata

FooterPartition

PackHeader

Metadata


Structural Header Metadata in MXFPrimer Pack KEY: NOTES:Local Tag Entry Batch Composition by For clarity, Instance UID and(for assignment of local tags) Strong Reference Generation UID properties are

HasA by ownership not shown in this diagram.

Preface 1..n Identification Aggregation by Last Modified Date This Generation UID General Weak ReferenceVersion Company Name HasA by sharingOperational Pattern (UL) Product NameEssence Containers (UL Batch) Version String 0..1 : 0 or 1DM Schemes (UL Batch) Product UID 1..1 : 1 and only 1

Modification Date 0..n : 0 or more1..n : 1 or more

1..1 Content Storage 1..n Essence Container Data Link 1..1 Partition with BodySIDBodySID EssenceLinked Package UID Container

Link to Source Package by UMID Data

1..n Material Package 1..1 Track (Timecode) 1..1 Sequence (Timecode) 1..1 Timecode ComponentPackage UID Track Number Data Definition (UL) Data Definition (UL - same as Sequence)Package Creation Date Track Name Duration Duration (= Duration in Sequence)Package Modified Date Edit Rate Rounded Timecode Timebase

Origin Start TimecodeDrop Frame

1..n Track (Picture/Sound/Data) 1..1 Sequence (Picture/Sound/Data) 1..n Source Clip (Picture/Sound/Data)Track Number Data Definition (UL) Data Definition (UL - same as Sequence)Track Name Duration DurationEdit Rate Start PositionOrigin Source Package ID

Source Track ID

Link to Source Package ID (UMID)Link to Source Track ID

1..n File Package (top-level) 1..n Track (Picture/Sound/Data) 1..1 Sequence (Picture/Sound/Data) 1..n Source Clip (Picture/Sound/Data)Package UID Track ID Data Definition (UL) Data Definition (UL - same as Sequence)Package Creation Date Track Number Duration DurationPackage Modified Date Track Name Start Position

Edit Rate Source Package ID = 0Origin Source Track ID = 0

1..n File Descriptors (as needed)


The MXF Source Reference Chain

Material Package(generally

describes the output timeline)

Sequence (defines duration)

SourceClip

Track (defines start)

Top-levelFile Packages(describe the

actual Essence in the file)


SourceClip SourceClip SourceClip

The Top-Level File Package SourceClip(s) may reference Lower-Level Source Packages. These do not describe actual stored Essence. They describe where the stored Essence came from e.g. previously conformed MXF files.


SourceClip


Body Container

SourceClipEssence Descriptor

e.g. Tape Descriptor

Essence Descriptore.g. MPEG

The Material Package SourceClip(s) reference the Top-Level File Package. This can be used to define an “EDL” of File Packages.


Lower-LevelSource Packages

(describe where the Essence

came from e.g. tape, reel,

source file)


How do DM Tracks work in MXF?

Picture TracksVideo, Images, Graphics

Sound TracksAudio, MIDI, Synthetic

Data TracksTele-text, Data Services, Closed Captions

Compound TracksFor intimately interleaved essence such as DV-DIF packets

DIF packet stream comprises video, audio, aux and other packetsMetadata Tracks

TimecodeDescriptive metadata


What DM Tracks kinds can be used?

Static TracksHave: no origin, no durationHave: a single metadata segment/source-clip

no start, no durationTimeline Tracks

Have: origin & durationHave: a sequence of 1 or more contiguous segments/source-clips

Each segment has duration where:sum of segment/source-clip durations = track duration

Event TracksHave: event origin & (optional) durationHave: 1 or more segments/source-clips which are not contiguous

Segments/source-clips may be intermittent along the timelineEach segment/source-clip has a start point and duration


Structural Header Metadata in MXFMaterial Package File Package (top-level) 1..1 Tracks (as needed)Package Creation Date Package Creation DatePackage Modified Date Package Modified Date

1..1 File Descriptors (as needed)1..1 Track (DM) [Note 2] 1..1 Track (DM) [Note 2]

Track Number Track NumberTrack Name Track NameEdit Rate Edit RateOrigin Origin NOTES:

1. For clarity, Instance UID and Generation UID properties are not shown in this diagram.

1..1 Sequence (DM) [Note 3] 1..1 Sequence (DM) [Note 3] 2. If the DM Track is static, the 'origin' Data Definition (UL) Data Definition (UL) property is not usedDuration Duration 3. If the DM Track is static, the 'duration'

property is not used in the Sequence and.DM Segment sets.

1..1 DM Segment [Note 3] 1..1 DM Segment [Note 3]Data Definition (UL - same as Sequence) Data Definition (UL - same as Sequence)Duration (= Duration in Sequence) Duration (= Duration in Sequence)

Structural Metadata (SMPTE 377M)

1..1 1. Production Framework 1..1 2. Clip Framework Descriptive Metadata (SMPTE 380M)Framework Extended Text Language Code Framework Extended Text Language CodeFramework Title Framework TitlePrimary Extended Spoken Language Code Primary Extended Spoken Language Code Integration Indication Clip Kind KEY:

Clip Number Composition byExtended Clip ID Strong Reference

1..1 4. Titles Clip Creation Date & Time HasA by ownershipMain Title Take NumberSecondary Title Slate Information Aggregation by Working Title General Weak ReferenceOriginal Title HasA by sharingVersion Title 1..1 4. Titles

Main Title 0..1 : 0 or 11..1 5. Identification Secondary Title 1..1 : 1 and only 1

Identifier Kind Working Title 0..n : 0 or moreIdentifier Value Original Title 1..n : 1 or moreIdentification Locator Version TitleIdentification Issuing Authority

To other sets as needed To other sets as needed


DM Segment or DM Source Clip?

DM SegmentDefines the metadata to describe the associated essenceLinks to an essence track in the same PackageMay be present in a Material Package (top level)

To define DM for the file outputMay be present in a Top-level File Package

To define DM for the essence in the MXF essence containerDM Source Clip

Uses the ‘SourceClip’ mechanism used by essence tracksAllows metadata to be referenced from a DM Segment

In a lower level Source Package


How does DM Segment work?

Timecode TrackPicture Track

Virtual DMTrack

Sound Track 2Sound Track 1

Root Sets(Preface,Ident &ContentStorage)

Package(Material,File andSource)

DMSegment 1

Edit unit

DM Track&

Sequence

Data Track

HeaderMetadata

Links to all essence tracks (default) Links only to sound tracks Links only to the picture track

DMFramework

DMSegment 2

DMFramework

DMSegment 3

DMFramework

EventStart

Position

Duration


How does DM Source Clip work?


DM Track


Root Sets(Preface,Ident &ContentStorage)

MaterialPackage

DMSegment

Edit unit

DM Track &Sequence

Data Track

HeaderMetadata

Links to

DMFramework

SourcePackage

DM Track&

Sequence

DM SourceClip

SourcePackageID

SourceTrack ID

Links to

Start Position Duration

DurationEventStart

Position


DM Track


Data TrackLinks to

Origin

Origin

Material(Output)

Source(Input)


Summary

MXF Structural Metadata defines the timeline…For essenceFor metadata

MXF Structural Metadata defines the DM plug-inAll DM is owned by a metadata track

The Metadata track may:Reference other DM Tracks in another Package

Via a DM Source Clip

Define the DM through a DM SegmentA DM Segment

Owns one DM FrameworkThe DM Framework may be from any MXF DM Scheme


Essential Reading (see the book!)

SMPTE 377M (MXF Format)EG41 (Primary MXF engineering guideline)EG42 (DM for MXFSMPTE 380M (DMS-1)