Introduction Differential Privacy under Dependent Data Conclusion and Future Work Dependence Makes You Vulnerable: Differential Privacy Under Dependent Tuples Changchang Liu 1 , Supriyo Chakraborty 2 , Prateek Mittal 1 Email: 1 {cl12, pmittal}@princeton.edu, 2 [email protected], 1 Princeton University, 2 IBM T.J. Watson Research Center February 23, 2016 1 / 21
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Dependence Makes You Vulnerable: DifferentialPrivacy Under Dependent Tuples
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Inference Attack on DP via K-Means Query
Differentially Private K -means for Gowalla Location Dataset
Individuals Data Provider
Raw DataK-means
Clustering
Data Recipients
Differentially Private
K-means Clustering
Perturbation
Inference Attack
12 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Inference Attack
Social
RelationshipsInference Attack
Check-in
Community
13 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Inference results by using correlation
0 0.5 1 1.5 2 2.5 30
2
4
6
8
Privacy Budget ε
Lea
ked
In
form
atio
n
with social relationships
w/o social relationships
security guarantee by DP
Exploiting correlation, one can infer more information!Exploiting correlation can break DP security guarantees!
14 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Inference results by using correlation
0 0.5 1 1.5 2 2.5 30
2
4
6
8
Privacy Budget ε
Lea
ked
In
form
atio
n
with social relationships
w/o social relationships
security guarantee by DP
Exploiting correlation, one can infer more information!Exploiting correlation can break DP security guarantees!
14 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Differential Privacy under Dependent DataInference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
14 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
ε-Dependent Differential Privacy (DDP)
Neighboring Databases
•R is probabilistic dependence relationship among the L dependent tuples•The adversary’s ability to infer the individual’s information is boundedeven if the adversary has access to data correlation R.
15 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
ε-Dependent Differential Privacy (DDP)
Neighboring
Databases
Dependent Differential
Privacy requires:
•R is probabilistic dependence relationship among the L dependent tuples•The adversary’s ability to infer the individual’s information is boundedeven if the adversary has access to data correlation R.
15 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
ε-Dependent Differential Privacy (DDP)
Probability
S Query Output
Neighboring
Databases
Dependent Differential
Privacy requires:
•R is probabilistic dependence relationship among the L dependent tuples•The adversary’s ability to infer the individual’s information is boundedeven if the adversary has access to data correlation R.
15 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Dependent Perturbation Mechanism
• Augment conventional LPM with additional noise relevant to ρij
• Dependent coefficient ρij
− extent of dependence of Dj on the modification of Di
16 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Dependent Coefficient
Laplace noise in dependent perturbation mechanism
exp
{− ε
Sensitivityi + ρij ×Sensitivityj
}Dependent coefficient satisfies: 0≤ ρij ≤ 1
• ρij = 0: standard differential privacy (independent setting)
• ρij = 1: fully dependent setting
• ρij : formulate correlation from privacy perspective
17 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Limitations of Dependent Coefficient
The exact computation of ρij
relies on knowledge of data generation model
18 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Differential Privacy under Dependent DataInference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
18 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Resilience to Inference Attack
0 0.5 1 1.5 2 2.5 30
2
4
6
8
Privacy Budget ε
Lea
ked
In
form
atio
n
attack DPattack DDP
DDP is more resilient to inference attack
19 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Resilience to Inference Attack
0 0.5 1 1.5 2 2.5 30
2
4
6
8
Privacy Budget ε
Lea
ked
In
form
atio
n
attack DPattack DDP
DDP is more resilient to inference attack
19 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Inference Attack for DP based on Correlated TuplesDependent Differential Privacy (DDP)Experimental Results
Further Analysis and Experiments
• Composition Property
− Sequential/parallel composition property
• Theoretical utility analysis
• Different classes of queries
− Machine learning queries− Graph queries
20 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Conclusion and Future work
• Incorporate correlation into differential privacy
− Dependent differential privacy− More resilient to inference attack
• Alternative data generation models in the future work
21 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Appendix1: Dependence between tuples can seriouslydegrade the privacy guarantees provided by the existing DPmechanisms
QuerySum
,i jD Dé ùë û ( )1Lap e
Add noise Noisy ( )i jD D+
Independent
Privacy
Guarantee( )exp e
( )exp 1.5eDependent
i jD D^
Smaller means better privacy e
[ ]0.5 0.5
0,1
j i
i
D D X
D X
= +
and are i.i.d in
Privacy
Guarantee
21 / 21
IntroductionDifferential Privacy under Dependent Data
Conclusion and Future Work
Appendix 2: Model to Compute Dependent Coefficient
Here, we consider to utilize the friend-based model to compute theprobabilistic dependence relationship, where a user’s location can beestimated by her friend’s location based on the distance between theirlocations. Specifically, the probability of a user j locating at dj whenher friend i is locating at di is