Learning Adversarially Fair and Transferrable Representations David Madras \[ | {z } me Elliot Creager \[ Toniann Pitassi \[ Richard Zemel \[ \ University of Toronto [ Vector Institute July 13, 2018 Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 1 / 18
18
Embed
Learning Adversarially Fair and Transferrable Representationsmadras/presentations/laftr-talk-icml.pdf · 2018-07-21 · Learning Adversarially Fair and Transferrable Representations
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Learning Adversarially Fair and TransferrableRepresentations
David Madras\[| {z }me
Elliot Creager\[ Toniann Pitassi\[ Richard Zemel\[
\University of Toronto
[Vector Institute
July 13, 2018
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 1 / 18
[Dwork et al., 2012] [Hardt et al., 2016]Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 6 / 18
Goals of Fair Representation Learning
Fair classification: learn X
f! Z
g! Y
encoder f , classifier g
Fair representation: learn X
f! Z
g! Y
Z = f (X ) should:Maintain useful information in X
Yield fair downstream classification for vendors g
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 7 / 18
Types of unfair vendors
Consider two types of unfair vendorsThe indi↵erent vendor: doesn’t care about fairness, only maximizesutilityThe malicious vendor: doesn’t care about utility, discriminatesmaximally
This suggests an adversarial learning scheme
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 8 / 18
Learning Adversarially Fair Representations
AZY
X
Encoderf (X )
Decoderk(Z ,A)
Classifierg(Z )
Adversaryh(Z )
The classifier is the indi↵erent vendor, forcing the encoder to makethe representations useful
The adversary is the malicious vendor, forcing the encoder to hide thesensitive attributes in the representations
[Edwards and Storkey, 2015]Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 9 / 18
Adversarial Learning in LAFTR
AZY
X
Encoderf (X )
Decoderk(Z ,A)
Classifierg(Z )
Adversaryh(Z )
Our game: encoder-decoder-classifier vs. adversary
Goal: learn a fair encoder
minimizef ,g ,k
maximizeh
EX ,Y ,A [L(f , g , h, k)] .
L(f , g , h, k) = ↵LClass + �LDec � �LAdv
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 10 / 18
Adversarial Objectives
AZY
X
Encoderf (X )
Decoderk(Z ,A)
Classifierg(Z )
Adversaryh(Z )
Choice of adversarial objective depends on fairness desideratum
Demographic parity: LDPAdv (h) =
Pi2{0,1}
1
|Di |P
(x ,a)2Di|h(f (x)) � a|
Equalized odds: LEOAdv (h) =
Pi ,j2{0,1}2
1
|Dji |
P(x ,a,y)2Dj
i|h(f (x), y)� a|
Equal Opportunity: LEOppAdv (h) =
Pi2{0,1}
1
|D1
i |P
(x ,a)2D1
i|h(f (x)) � a|
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 11 / 18
From Adversarial Objectives to Fairness Definitions
In general: pick the right adversarial loss, encourage the right conditionalindependencies
Demographic parity encourages Z ? A to fool adversary
Equalized odds encourages Z ? A | Y to fool adversary
Equal opportunity encourages Z ? A | Y = 1 to fool adversary
Note that independencies of Z = f (x) also hold for predictions Y = g(Z )
We show: In the adversarial limit, these objectives guarantee thesefairness metrics!
The key is to connect predictability of A by the adversary h(Z ) tounfairness in the classifier g(Z )
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 12 / 18
Theoretical Properties
Define �DP(g) , DP-unfairness of classifier g
Define LDPAdv (h) , adversarial loss (inv. weighted error)
We show: 8 classifier g(Z ), we can construct an adversary h(Z ) s.t.�LDP
Adv (h) = �DP(g)
Let h? be the optimal adversary. Then
�LDPAdv (h
?) � �LDPAdv (h) = �DP (1)
Takeaway: if �LDPAdv (h
?) is forced to be small, �DP will be too
Holds for EO as well, but with h as a function of Y also
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 13 / 18
Results - Fair Classification (Adult)
0.025 0.050 0.075 0.100 0.125 0.150 0.175�DP
0.80
0.81
0.82
0.83
0.84
0.85
Acc
urac
y
LAFTR-DP
LAFTR-EO
LAFTR-EOpp
DP-CE
MLP-Unfair
0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16�EO
0.838
0.840
0.842
0.844
0.846
0.848
0.850
Acc
urac
y
LAFTR-DP
LAFTR-EO
LAFTR-EOpp
MLP-Unfair
0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08�EOpp
0.836
0.838
0.840
0.842
0.844
0.846
0.848
0.850
Acc
urac
y
LAFTR-DP
LAFTR-EO
LAFTR-EOpp
MLP-Unfair
Train with two-step method to simulate owner ! vendor framework
Tradeo↵s between accuracy and various fairness metrics yielded bydi↵erent LAFTR loss functions
Seems to work best for fairest solutions
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 14 / 18
Setup - Fair Transfer Learning
Downstream vendors will have unknown prediction tasks
Does fairness transfer?
We test this as follows:1 Train encoder f on data X , with label Y2 Freeze encoder f3 On new data X
0, train classifier on top of f (X 0), with new task label Y 0
4 Observe fairness and accuracy of this new classifier on new task Y
0
Compare LAFTR encoder f to other encoders
We use Heritage Health datasetY is Charlson comorbidity index > 0Y
0 is whether or not a certain type of insurance claim was madeCheck for fairness w.r.t. age
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 15 / 18
Results - Fair Transfer Learning
Transfer-Unf Transfer-Fair Transfer-Y-Adv LAFTR
�0.4
�0.3
�0.2
�0.1
0.0
0.1
Rel
ativ
eD
i�er
ence
toba
selin
e(T
arge
t-U
nfai
r)
Error�EO
Figure 2: Fair transfer learning on Health dataset. Down is better in both metrics.
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 16 / 18
Conclusion
Propose LAFTR: general model for fair representation learning
Connect common fairness metrics to adversarial objectives
Demonstrate that training with LAFTR improves transfer fairness
Open questions:Compare adversarial/non-adversarial methods?Transfer fairness: datasets, limitations, better methods?
Come check out our poster #44 tonight!
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 17 / 18
References
Dwork, C., M. Hardt, T. Pitassi, O. Reingold, and R. Zemel (2012).Fairness through awareness. In Proceedings of the 3rd innovations in
theoretical computer science conference, pp. 214–226. ACM.
Edwards, H. and A. Storkey (2015). Censoring representations with anadversary. arXiv preprint arXiv:1511.05897 .
Hardt, M., E. Price, N. Srebro, et al. (2016). Equality of opportunity insupervised learning. In Advances in neural information processing
systems, pp. 3315–3323.
Zemel, R., Y. Wu, K. Swersky, T. Pitassi, and C. Dwork (2013). Learningfair representations. In International Conference on Machine Learning,pp. 325–333.
Madras et al. 2017 (arxiv:1802.06309) LAFTR: Poster #44 July 13, 2018 18 / 18