YCoCg: A Color Space with RGB Reversibility (FRExt) 1 st Feb 2006.

YCoCg:A Color Space

with RGB Reversibility(FRExt)

1st Feb 2006

H.264 has four profiles:

• Baseline (IP Video phone, Simple streaming)

• Main Profile (Digital Storage Media, Television Broadcasting)

• Extended Profile (Streaming video)

• High Profile (Content contribution, Content distribution, Studio editing, post processing)

Profiles

There are four High Profiles (Fidelity range extensions-FRExt)

• High Profile is to support the 8-bit video with 4:2:0 sampling for applications using high resolution.

• High 10 Profile is to support the 4:2:0 sampling with up to 10 bits of representation accuracy per sample.

• High 4:2:2 Profile is to support up to 4:2:2 chroma sampling and up to 10 bits per sample.

• High 4:4:4 Profile is to support up to 4:4:4 chroma sampling, up to 12 bits per sample, and integer residual color transform for coding RGB signal.

Mainly targeting the HD DVD, Broadcast, Editing, Digital still camera market

High Profile

Specific coding parts for the Profiles

MacroBlock (MB) structure in 4:4:4 format

4:4:4 MB

Y Cb Cr

16

161616

In 4:4:4 video, luma and both the chroma componentswill have same spatial resolution

Applications using High Profile

• Use more than 8 bits per sample of source video accuracy• Use higher resolution for color representation than what is typical in

consumer applications (i.e., 4:2:2 or 4:4:4 sampling as opposed to 4:2:0 chroma sampling format)

• Perform source editing functions such as alpha blending (a process for blending of multiple video scenes, best known for use in weather reporting where it is used to super- impose video of a newscaster over video of a map or weather-radar scene)

• Use very high bit rates

• Use very high resolution

• Achieve very high fidelity – even representing some parts of the video losslessly

• Avoid color-space transformation rounding error

• Use RGB color representation

High Profile

H.264 Encoder

• In 4:4:4 video, FRExt has Residual Color Transform.

• Keep RGB domain (same depth) for input, output and stored reference pictures and use the forward and inverse color transformations inside the encoder and decoder for processing of the residual data only.

• Eliminates color-space conversion error without significantly increasing the overall complexity of the system.

H.264 Encoder

= Residual Color Transform RGB to YCoCg

Residual color transform exploits the redundancy among the residual data of each RGB component after intra/inter prediction. The correlation among the color components still exists even after the intra/inter prediction. To decorrelate this redundancy, we apply the YCoCg color transform to the residual data.

This scheme can be easily incorporated into the existing JVT specification.

This technique achieves better coding efficiency than applying the color transform outside the coding loop as manifested in the simulation results of [1], [2].

YCoCg color space has improved coding gain relative to both RGB and YCbCr.

Y = Luminance componentCo = Orange Chroma componentCg = Green Chroma component

There are two problems with this approach.

1. Since the samples are actually represented using integers, rounding error is introduced in both the forward and inverse color transformations.

2. The second is that, because the above transformation was not originally designed for digital video compression, it uses a sub-optimal trade-off between the complexity of the transformation (with difficult-to-implement coefficient values such as 0.2126 and 0.0722) and coding efficiency.

YCbCr Color Space

Forward Color Space Transform

Inverse Color Space Transform

YCoCg Color Space

So encoder/decoder needs only additions and shifts to RGB YCoCg

+1- 1/2

-1 1/2

ΔG

-1 1/2

ΔR

ΔB

ΔCo

t

ΔCg

ΔY

FORWARD TRANSFORM

+1- 1/2

ΔG

ΔB

ΔR

t

REVERSE TRANSFORM

ΔCo

ΔCg

ΔY

+1- 1/2

-1 1/2

ΔG

-1 1/2

ΔR

ΔB

ΔCo

t

ΔCg

ΔY

FORWARD TRANSFORM

+1- 1/2

ΔG

ΔB

ΔR

t

REVERSE TRANSFORM

ΔCo

ΔCg

ΔY

+1- 1/2

-1 1/2

ΔG

-1 1/2

ΔR

ΔB

ΔCo

t

ΔCg

ΔY

FORWARD TRANSFORM

+1- 1/2

ΔG

ΔB

ΔR

t

REVERSE TRANSFORM

ΔCo

ΔCg

ΔY

R, G, and B are the residual data andY, Co, and Cg are the residue transformed data

Simulation Results that show the coding gain for Y component in dB

Seq.

bit rate (Mbps)

10 40 100

Analog TV NA 1 1.5

Trees and Bicycle 0.5 0.6 0.8

Man in Restaurant 0.5 0.7 1

Rolling Tomatoes 0.5 1.2 1.2

Card Toss NA 1 1.2

Dinner NA 1 1.2

The coding gain using residual color transform for Y component in YCoCg color domain is about 0.5~1.5 db depending on the bit rate.

Note: These results are taken from [2]

Conclusion

YCoCg color space transformation has three important features:

1. Exactly reversible in integer arithmetic. (i.e MSE=0)2. Minimal increase in dynamic range; no increase for Y and only 1 bit increase for Co and Cg.3. Higher coding gain than other color spaces.

RGB Component

YCoCg Component

Reconstructed RGB Component

References

[1] JVT document - JVT-I014r3.doc (ftp://standards.polycom.com)

[2] JVT document - JVT-L025.doc

[3] JVT document - JVT-Q059_L.doc & JVT-Q059_L.xls

[4] Soon-kak Kwon, A. Tamhankar, K. R. Rao, “Overview of H.264/MPEG-4 Part10”, Special issue on “ Emerging H.264/AVC video coding standard”, J. Visual Communication and Image Representation, vol.17, 2006

[5] G. Sullivan, P. Topiwala and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” SPIE Conference on Applications of Digital Image Processing XXVII, vol. 5558, pp. 53-74, Aug. 2004.

[6] H.264 : International Telecommunication Union, “Recommendation ITU-T H.264: Advanced Video Coding for Generic Audiovisual Services,” ITU-T, 2003

Thank You