Contribution of Non-Scrambled Chroma Information in Privacy-Protected Face Images to Privacy Leakage Hosik Sohn 1, Dohyoung Lee 2, Wesley De Neve 1, Konstantinos.

Contribution of Non-Scrambled Chroma Information in Contribution of Non-Scrambled Chroma Information in Privacy-Protected Face Images to Privacy LeakagePrivacy-Protected Face Images to Privacy Leakage

Contribution of Non-Scrambled Chroma Information in Contribution of Non-Scrambled Chroma Information in Privacy-Protected Face Images to Privacy LeakagePrivacy-Protected Face Images to Privacy Leakage

Hosik Sohn1, Dohyoung Lee2, Wesley De Neve1, Konstantinos N. Plataniotis2, and Yong Man Ro1

1Korea Advanced Institute of Science and Technology (KAIST),Image and Video Systems Lab.

2University of Toronto, Multimedia Lab

October 2011.

10th International Workshop on Digital-forensics and Watermarking (IWDW 2011)

ContentsContents--22--

1. Introduction1. Introduction1. Introduction1. Introduction

2. Layered Scrambling for Motion JPEG XR2. Layered Scrambling for Motion JPEG XR2. Layered Scrambling for Motion JPEG XR2. Layered Scrambling for Motion JPEG XR

3.1 Objective Assessments3.1 Objective Assessments3.1 Objective Assessments3.1 Objective Assessments

3.2 Subjective Assessments3.2 Subjective Assessments3.2 Subjective Assessments3.2 Subjective Assessments

3. Assessment of Chroma-induced Privacy Leakage3. Assessment of Chroma-induced Privacy Leakage3. Assessment of Chroma-induced Privacy Leakage3. Assessment of Chroma-induced Privacy Leakage

4. Discussion and Conclusion4. Discussion and Conclusion4. Discussion and Conclusion4. Discussion and Conclusion

INTRODUCTIONINTRODUCTION

--33--

1. Introduction1. Introduction

Present-day video surveillance systems often come with high-speed network connections, plenty of storage capacity, and high processing power

The increasing ability of video surveillance systems to identify people has recently raised several privacy concerns

To mitigate these privacy concerns, scrambling can be leveraged to conceal the identity of face images in video content originating from surveillance cameras

--44--

Privacy protected surveillance videos

1. Introduction1. Introduction

The past few years have witnessed the development of a wide range of content-based tools for protecting privacy in video surveillance systems

Dependent on the location where scrambling (or encryption) is applied, three different approaches of scrambling can be distinguished

Uncompressed domain scrambling Transform domain scrambling Compressed bit stream domain scrambling

One of the main challenge is to concealment of privacy-sensitive regions by making use of invertible transformation of visual information at a low computational cost

--55--

1. Introduction1. Introduction

Content-based tools for privacy protection need to find a proper balance between the level of security offered and the amount of bit rate overhead

In general, altering the visual information present in privacy-sensitive regions typically breaks the effectiveness of coding tools

To limit bit rate overhead, many content-based tools for privacy protection only scramble luma information, leaving chroma information unprotected

--66--

Coding efficiency

Security level

1. Introduction1. Introduction

In this paper, we investigate the contribution of non-scrambled chroma information to privacy leakage

To that end, we study and quantify the influence of the presence of non-scrambled chroma information on the effectiveness of automatic and human FR

Objective assessment: we apply automatic FR techniques to face images have been privacy-protected in the luma domain

Subjective assessment: we investigate whether agreement exists between the judgments of 32 human observers and the output of automatic FR

--77--

1. Introduction1. Introduction

FR vs. Perception-based security metrics for assessing privacy level

Luminance Similarity Score (LSS), Edge Similarity Score (ESS), and Local Feature-based Visual Security Metric (LFVSM)[1,2]

However, these metrics are general in nature and are thus not able to take advantage of domain-specific information (e.g., face information)

--88--

[1] Tong, L., Dai, F., Zhang, Y., Li, J. “Visual security evaluation for video encryption,” in: Proceedings of ACM International Conference on Multimedia, 835–838 (2010)[2] Mao Y., Wu M., "A joint signal processing and cryptographic approach to multimedia encryption," IEEE Transactions on Image Processing, 15(7), (2006), 2061-2075.

LAYERED SCRAMBLING FORLAYERED SCRAMBLING FORMOTION JPEG XRMOTION JPEG XR

--99--

2. Layered scrambling for Motion JPEG XR2. Layered scrambling for Motion JPEG XR

The video surveillance system studied makes use of Motion JPEG XR to encode surveillance video content Motion JPEG XR offers a low-complexity solution for the intra coding of high-resolution

video content, while at the same time offering quality and scalability provisions

Layered scrambling for JPEG XR [3]

Modified JPEG-XR encoder

--1010--

LBT Q Pred.LBT

Adaptive scan

Q Pred.

Adaptive scan

Q Pred.

Scrambling(RLS) • Adaptive

entropy coding

• Fixed length coding

DC subband

LP subband

HP subband/Flexbits

Scrambling(RP)

Scrambling(RSI)

Secret key

[3] Sohn, H., De Neve, W., Ro, Y.M., “Privacy Protection in Video Surveillance Systems: Analysis of Subband-Adaptive Scrambling in JPEG XR,” IEEE Transactions on Circuits and Systems for Video Technology, 21, 170–177 (2011)

2. Layered scrambling for Motion JPEG XR2. Layered scrambling for Motion JPEG XR

Overview of layered scrambling technique

N denotes the number of MBs, L denotes the RLS parameter, K denotes the number of non-zero LP coefficients in a MB, and M denotes the number of non-zero HP coefficients in a MB.

--1111--

),(LRDCcoeffDCcoeff e

, ,..., , ,...,1 , 1 Cje

i xxjCiwhereLPcoeffLPcoeff

, ,

1 ,

otherwiseHPcoeff

rifHPcoeffHPcoeff e

- Random level shift (RLS) for DC subbands

- Random permutation (RP) for LP subbands

- Random sign inversion (RSI) for HP subbands

ASSESSMENT OF CHROMA-INDUCED ASSESSMENT OF CHROMA-INDUCED PRIVACY LEAKAGEPRIVACY LEAKAGE

--1212--

3.1. Objective assessments3.1. Objective assessments

Experimental setup FR technique used: PCA, FLDA, LBP Face images: 3070 frontal face images of 68 subjects from CMU PIE

(68 gallery, 340 training, and 2662 probe face images) Probe face images represent privacy-protected face images that appear in video

content originating from surveillance cameras.

Performance evaluation: Cumulative Match Characteristic (CMC) curve

Notations

--1313--

Notation Explanation

DC, LP, and HP DC, LP, and HP subband

S3 DC+LP+HP

S2 DC+LP

S1 DC

Subscripts (Y, Co, Cg) Luma and chroma channels (Y, Co, and Cg)

Prime (′) Scrambled image data

3.1. Objective assessments3.1. Objective assessments

Influence of distance measurement on FR effectiveness Distance metric: Euclidean, Mahalanobis, Cosine, and Chi-square distance

In the remainder of our experiments, we make use of the Euclidean distance metric for PCA- and FLDA-based FR, and the Chi-square distance metric for LBP-based FR

--1414--

DE : Euclidean distanceDM : Mahalanobis distanceDC : Cosine distanceDH : Chi-square distance

3.1. Objective assessments3.1. Objective assessments

Scrambled luma information Assumes that an adversary is not able to take advantage of the possible presence

of non-scrambled chroma information in the privacy-protected probe face images

--1515--

3.1. Objective assessments3.1. Objective assessments

Scrambled luma and non-scrambled chroma information We investigate whether layered scrambling is still effective when the

scrambled luma channel and the non-scrambled chroma channels are simultaneously used for the purpose of automatic FR

Assuming that an adversary has access to the compressed bit stream structure, and thus to the non-scrambled chroma information

To take advantage of non-scrambled chroma information, we adopted feature-level fusion

--1616--

3.1. Objective assessments3.1. Objective assessments

Scrambled luma and non-scrambled chroma information

--1717--

3.1. Objective assessments3.1. Objective assessments

Non-scrambled chroma information

--1818--

3.2. Subjective assessments3.2. Subjective assessments

Experimental setup Number of observer: 32 We presented three scrambled probe face images of different subjects to the

human observers for each experimental condition Assessment method

Human observers were asked to select the gallery face image that is most similar to the given probe face image

human observers were also able to study the probe face images at different zoom levels

--1919--

Gallery face images use for subjective assessment

3.2. Subjective assessments3.2. Subjective assessments

Non-scrambled chroma information

--2020--

3.2. Subjective assessments3.2. Subjective assessments

Scrambled luma and non-scrambled chroma information

--2121--

DISCUSSION & CONCLUSIONDISCUSSION & CONCLUSION

--2222--

4. Discussion4. Discussion

For video surveillance applications requiring a high level of privacy protection, both the luma and the chroma channels need to be scrambled at the cost of a higher bit rate overhead

Layered scrambling to both the luma (Y) and the chroma channels (Co and Cg)

--2323--

4. Discussion4. Discussion

Bit rate overhead

Security (ideal case) Sub-sampling decreases the level of privacy protection, given the lesser amount

of data available for scrambling Total number of combinations required to break the protection of 10 MBs

reduces from 3.6×10722 (4:4:4) to 1.7×10360 (4:2:0)

--2424--

5. Conclusions and future work5. Conclusions and future work

This paper studied and quantified the influence of non-scrambled chroma infor-mation on the effectiveness of automatic and human FR

Our results show that, when an adversary has access to the coded bit stream structure, the presence of non-scrambled chroma information may significantly contribute to privacy leakage

For video surveillance applications requiring a high level of privacy protection, our results indicate that both luma and chroma information needs to be scrambled at the cost of an increase in bit rate overhead

In order to compile a benchmark for privacy protection tools, future research will focus on identifying additional worst case scenarios

--2525--

Thank you for your attention!

--2626--

APPENDIXAPPENDIX

Effectiveness of general-purpose visual security metrics Visual security metric used

Luminance Similarity Score (LSS), Edge Similarity Score (ESS), and Local Feature-based Visual Security Metric (LFVSM)

Lower the values computed by the visual security metrics, the higher the visual security

--2727--