Counterfactual Reasoning in Algorithmic Fairnessfairware.cs.umass.edu/slides/silva.pdf · Counterfactual Reasoning in Algorithmic Fairness Ricardo Silva University College London

CounterfactualReasoninginAlgorithmicFairness

RicardoSilvaUniversityCollegeLondonand

TheAlanTuringInstitute

JointworkwithMattKusner(Warwick/Turing),ChrisRussell(Sussex/Turing),andJoshuaLoftus(NYU)

FairnessandMachineLearning

•  Thedream:ifweteachmachinestoperformsensitivedecisions,theywillnotsufferfromhumanbiases.

•  Thereality:theGIGOprinciplestillholds,regardlessofwhetherwearetalkingofstatisticalmodelsorsoftware.

TheMessage

•  Thereisonlysomuchdataalonecantellyouaboutfairness.

•  I’mnottalkingabout“just”valuejudgments.

•  Weshouldhighlighttherolethatthedata-generatingcausalprocesshasinshapingournotionsoffairness.

NobodyisSayingThisisEasy

•  AtnopointIwillsuggestthatbuildingacausalmodeliseasy.

•  Someuntestedanduntestableassumptionswillbeneeded.

•  Theideaistomakeyourassumptionsasexplicitaspossible,hopefullybeing“lesswrong”intheend.

TheScopeofthisTalk

•  Weconsiderpredictionandinterventionproblems(moreoftheformer).

•  Inpredictionproblems,wewillhave:

X : features, or attributes of an individual

A : the protected attributes of an individual

Y : the target, what we would like to predict

Y : our prediction<latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="jgC96ZbtUjoycXYPDL+iaX6PzEU=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhK9gLitMCF4yLR3UJTVRNn0lh17MiebKmiSBxAiCs/jRu/gr+A043QdneZ09Ob5+eZN2mlpKMo+h2EV65eu35j7+bg1u07d+8N7z84cqa2AifCKGOnKThUUuOEJCmcVhahTBUep6u3Xf/4BK2TRn+gTYXzEpZa5lIAeWox/DN9xZ/yhPAz2bLhCZZV0eQIVFt07XNuLAciK9Oa0HGTc9Bc6kyeyKwG1SbJ4PWOARXYm1TWEArC7IxBe6nDx/84ENglkh9iXQDxNfK1qVXGlVwhJ8P9npkU1DkknWDXxgf0b5Ctzq/btoPFcBSNo23xiyDuwYj1dbgY/koyI+oSNQkFzs3iqKJ5A5akUNgOktphBWIFS5x5qKFEN2+2h2n5E89kPPch5kYT37JnXzRQOrcpU68sgQp3vteRl/VmNeUv543UlU9Vi9OP8lp1sXRX5pm0Pnu18QCElX5WLgqw4O9hXRdCfH7li+BofxxH4/h9NDp408exxx6xx+wZi9kLdsDesUM2YSKYBE3wNfgWfgq/hN/DH6fSMOjfPGQ7Ff78C0xY5Lo=</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="jgC96ZbtUjoycXYPDL+iaX6PzEU=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhK9gLitMCF4yLR3UJTVRNn0lh17MiebKmiSBxAiCs/jRu/gr+A043QdneZ09Ob5+eZN2mlpKMo+h2EV65eu35j7+bg1u07d+8N7z84cqa2AifCKGOnKThUUuOEJCmcVhahTBUep6u3Xf/4BK2TRn+gTYXzEpZa5lIAeWox/DN9xZ/yhPAz2bLhCZZV0eQIVFt07XNuLAciK9Oa0HGTc9Bc6kyeyKwG1SbJ4PWOARXYm1TWEArC7IxBe6nDx/84ENglkh9iXQDxNfK1qVXGlVwhJ8P9npkU1DkknWDXxgf0b5Ctzq/btoPFcBSNo23xiyDuwYj1dbgY/koyI+oSNQkFzs3iqKJ5A5akUNgOktphBWIFS5x5qKFEN2+2h2n5E89kPPch5kYT37JnXzRQOrcpU68sgQp3vteRl/VmNeUv543UlU9Vi9OP8lp1sXRX5pm0Pnu18QCElX5WLgqw4O9hXRdCfH7li+BofxxH4/h9NDp408exxx6xx+wZi9kLdsDesUM2YSKYBE3wNfgWfgq/hN/DH6fSMOjfPGQ7Ff78C0xY5Lo=</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="jgC96ZbtUjoycXYPDL+iaX6PzEU=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhK9gLitMCF4yLR3UJTVRNn0lh17MiebKmiSBxAiCs/jRu/gr+A043QdneZ09Ob5+eZN2mlpKMo+h2EV65eu35j7+bg1u07d+8N7z84cqa2AifCKGOnKThUUuOEJCmcVhahTBUep6u3Xf/4BK2TRn+gTYXzEpZa5lIAeWox/DN9xZ/yhPAz2bLhCZZV0eQIVFt07XNuLAciK9Oa0HGTc9Bc6kyeyKwG1SbJ4PWOARXYm1TWEArC7IxBe6nDx/84ENglkh9iXQDxNfK1qVXGlVwhJ8P9npkU1DkknWDXxgf0b5Ctzq/btoPFcBSNo23xiyDuwYj1dbgY/koyI+oSNQkFzs3iqKJ5A5akUNgOktphBWIFS5x5qKFEN2+2h2n5E89kPPch5kYT37JnXzRQOrcpU68sgQp3vteRl/VmNeUv543UlU9Vi9OP8lp1sXRX5pm0Pnu18QCElX5WLgqw4O9hXRdCfH7li+BofxxH4/h9NDp408exxx6xx+wZi9kLdsDesUM2YSKYBE3wNfgWfgq/hN/DH6fSMOjfPGQ7Ff78C0xY5Lo=</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="jgC96ZbtUjoycXYPDL+iaX6PzEU=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhK9gLitMCF4yLR3UJTVRNn0lh17MiebKmiSBxAiCs/jRu/gr+A043QdneZ09Ob5+eZN2mlpKMo+h2EV65eu35j7+bg1u07d+8N7z84cqa2AifCKGOnKThUUuOEJCmcVhahTBUep6u3Xf/4BK2TRn+gTYXzEpZa5lIAeWox/DN9xZ/yhPAz2bLhCZZV0eQIVFt07XNuLAciK9Oa0HGTc9Bc6kyeyKwG1SbJ4PWOARXYm1TWEArC7IxBe6nDx/84ENglkh9iXQDxNfK1qVXGlVwhJ8P9npkU1DkknWDXxgf0b5Ctzq/btoPFcBSNo23xiyDuwYj1dbgY/koyI+oSNQkFzs3iqKJ5A5akUNgOktphBWIFS5x5qKFEN2+2h2n5E89kPPch5kYT37JnXzRQOrcpU68sgQp3vteRl/VmNeUv543UlU9Vi9OP8lp1sXRX5pm0Pnu18QCElX5WLgqw4O9hXRdCfH7li+BofxxH4/h9NDp408exxx6xx+wZi9kLdsDesUM2YSKYBE3wNfgWfgq/hN/DH6fSMOjfPGQ7Ff78C0xY5Lo=</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="h0vKyMv1txiryC/6SUOT3b8sZzo=">AAACzXicdZFNb9NAEIbX5quEAoErlxUViAOKbC4gTlRcOBaJtIE4isbrcbzKfli746aRZYkDCPH3uPEr+Aus0wg1bZnTq5nZd2eeyWslPSXJ7yi+cfPW7Tt7dwf39u8/eDh8tH/sbeMEjoVV1k1y8KikwTFJUjipHYLOFZ7ky/d9/eQUnZfWfKJ1jTMNCyNLKYBCaj78M3nLn/OM8IycbnmGuq7aEoEah757ya3jQORk3hB6bksOhktTyFNZNKC6LBsc7hhQhVuT2llCQVhcMOiudfj8HwcCt0AKQ6wqIL5CvrKNKriSS+RkedizkIJ6h6xv2LUJgP4NsukL63bdYD48SEbJJvhVkW7FAdvG0Xz4KyusaDQaEgq8n6ZJTbMWHEmhsBtkjccaxBIWOA3SgEY/azeH6fizkCl4GSCW1hDfZC++aEF7v9Z56NRAlb9c65PX1aYNlW9mrTR1oGrE+Udlo3os/ZV5IV1gr9ZBgHAyzMpFBQ7CPZzvIaSXV74qjl+N0mSUfkzYHnvCnrIXLGWv2Tv2gR2xMRPROGqjb9H3+Ev8Nf5xjiuOttwes52If/4FarPjeg==</latexit><latexit sha1_base64="jgC96ZbtUjoycXYPDL+iaX6PzEU=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhK9gLitMCF4yLR3UJTVRNn0lh17MiebKmiSBxAiCs/jRu/gr+A043QdneZ09Ob5+eZN2mlpKMo+h2EV65eu35j7+bg1u07d+8N7z84cqa2AifCKGOnKThUUuOEJCmcVhahTBUep6u3Xf/4BK2TRn+gTYXzEpZa5lIAeWox/DN9xZ/yhPAz2bLhCZZV0eQIVFt07XNuLAciK9Oa0HGTc9Bc6kyeyKwG1SbJ4PWOARXYm1TWEArC7IxBe6nDx/84ENglkh9iXQDxNfK1qVXGlVwhJ8P9npkU1DkknWDXxgf0b5Ctzq/btoPFcBSNo23xiyDuwYj1dbgY/koyI+oSNQkFzs3iqKJ5A5akUNgOktphBWIFS5x5qKFEN2+2h2n5E89kPPch5kYT37JnXzRQOrcpU68sgQp3vteRl/VmNeUv543UlU9Vi9OP8lp1sXRX5pm0Pnu18QCElX5WLgqw4O9hXRdCfH7li+BofxxH4/h9NDp408exxx6xx+wZi9kLdsDesUM2YSKYBE3wNfgWfgq/hN/DH6fSMOjfPGQ7Ff78C0xY5Lo=</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit><latexit sha1_base64="Gz8RXW10rUfhoCUp5eXuuMqZf3s=">AAAC2HicdVFNj9MwEHXC11K+Chy5WFQgDqhKEBKI0wIXjotEdwtNVU2cSWPVsSN7sqWKInEAIa78NG78Cv4CTjdC291lTk9vnp9n3qSVko6i6HcQXrp85eq1veuDGzdv3b4zvHvv0JnaCpwIo4ydpuBQSY0TkqRwWlmEMlV4lK7edv2jY7ROGv2BNhXOS1hqmUsB5KnF8M/0FX/ME8LPZMuGJ1hWRZMjUG3RtU+5sRyIrExrQsdNzkFzqTN5LLMaVJskg9c7BlRgb1JZQygIs1MG7YUOH//jQGCXSH6IdQHE18jXplYZV3KFnAz3e2ZSUOeQdIJdGx/Qv0G2Or9u2w4Ww1E0jrbFz4O4ByPW18Fi+CvJjKhL1CQUODeLo4rmDViSQmE7SGqHFYgVLHHmoYYS3bzZHqbljzyT8dyHmBtNfMueftFA6dymTL2yBCrc2V5HXtSb1ZS/nDdSVz5VLU4+ymvVxdJdmWfS+uzVxgMQVvpZuSjAgr+HdV0I8dmVz4PDZ+M4Gsfvn4/23/Rx7LEH7CF7wmL2gu2zd+yATZgIJkETfA2+hZ/CL+H38MeJNAz6N/fZToU//wJNmOS+</latexit>

PredictionProblems•  PredictionheremeansinferringapropertyYthatwillbeusedfordecisionmaking.

•  Forexample:–  Y=1means“thispersonwilldefaultonaloan”(forthedecision,“shouldIgivethispersonaloan”?)

–  Y=1means“thispersonwillcommitacrimeintwoyears”(forthedecision,“shouldIreleasethisconvictnow?”)

•  WewouldliketopredictYina“fair”way,meaningthatourpredictionsshouldnotbe“biased”againstparticularinstancesofA.

Primitives

•  EvenifwetakethechoiceofwhatgoesinAasaprimitive,itisstillnotobviouswhatwemeanbybeingfair.

•  Afirstidea:“ensurethatdoesnotuseA”.

•  Thisisknowntobeunsatisfactory.

Y<latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit><latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit><latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit><latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit>

Examples•  Equalizedodds:giventheoutcomeY,attributeAprovidesnofurtherinformationaboutmyprediction.

•  Calibration:givenmyprediction,attributeAprovidesnofurtherinformationabouttheoutcomeY.

•  IfAisonaverageinformativeofY,wecannotreconciletheabove.–  Remember,herewedonotcontrolY(directly).Wedecideonpredictor.Y

<latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit><latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit><latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit><latexit sha1_base64="M+ZGxSf9Wvacj2hahaLwRB9eAwM=">AAAB7XicbVDLSgNBEOz1GeMr6tHLYBA8hV0R9Bj04jGCeUiyhNnJbDJmdmaZ6RVCyD948aCIV//Hm3/jJNmDJhY0FFXddHdFqRQWff/bW1ldW9/YLGwVt3d29/ZLB4cNqzPDeJ1pqU0ropZLoXgdBUreSg2nSSR5MxreTP3mEzdWaHWPo5SHCe0rEQtG0UmNzoAieeiWyn7Fn4EskyAnZchR65a+Oj3NsoQrZJJa2w78FMMxNSiY5JNiJ7M8pWxI+7ztqKIJt+F4du2EnDqlR2JtXCkkM/X3xJgm1o6SyHUmFAd20ZuK/3ntDOOrcCxUmiFXbL4oziRBTaavk54wnKEcOUKZEe5WwgbUUIYuoKILIVh8eZk0ziuBXwnuLsrV6zyOAhzDCZxBAJdQhVuoQR0YPMIzvMKbp70X7937mLeuePnMEfyB9/kDEfyOyA==</latexit>



PuttingItintheContextofaCausalModel

•  Atoymodel:imagineAisrace,Xis“ownsaredcar”andYis“crashescarinoneyear”.

•  Let’s(informally)drawacausaldiagramshowingcause-effectrelationshipsamongthose.Itwillincludea“unobservedtrait”Umeasuringaggressiveness.– Wewillgetintomoreformaldefinitionslater.

ACausalDiagram

SomeInitialConclusions

•  AisnotacauseofY.

•  IfwebuildapredictorbasedonX,ittellsussomethingbothaboutAandaboutU.

•  Hence,ourpredictorwillbedifferentfordifferentvaluesofA,whichdoesnotseemappropriate.

ASecondCausalDiagram

WhichConclusions?

•  AisnowacauseofY(indirectly).

•  Itisnowimpossibletosatisfybothequalizedoddsandcalibrationsimultaneously.

•  Judgmentcall:isthepathwayAèXèY“fair”?

ZoomingIn,withAnotherExample

•  Aherestandsforrace,Yforloandefault.

•  Sameidea,augmentedwithamediator:

ACausalPrimitive:CounterfactualFairness

•  Ifwehavesomeprotectedattributelikerace,andadecisionsuchaslengthofsentence,thenourdecisionsatisfiescounterfactualfairnessif

•  Acausalmodelisnecessarytoinfersuchclaimsfromdata.

“hadtheprotectedattributes(e.g.,race)oftheindividualbeendifferent,otherthingsbeingequal,thedecisionwouldhaveremainedthesame”

Workflow

•  Regardlessofthemachinelearningalgorithmtobeused,workwithadomainexperttoestimateacausalmodelofyourdata.–  It’samodeloftheworld,notofyoursoftware.

•  Chooseanymachinelearningalgorithmofinterest,anyblack-boxthattakesasinputsobservedandunobservedvariablesinyourdomain.–  Selectasetofvariablesbasedonwhichsetsrespectcounterfactualfairness.

–  Ifnecessary,inferunobservedvariablesfromtheobservedones.

FormalizingtheIdea

•  FormalnotionsofcounterfactualsdatebackatleasttoJerzyNeymaninthe1920s.

•  IwillfollowmostlytheStructuralCausalModel(SCM)frameworkofJudeaPearl,whichhascloselinkstotheworkofJamesRobins,andthatofSpirtes,GlymourandScheines.

StructuralCausalModels

•  Adirectedacyclicgraph(DAG)postulates“directcause-effect”pairs.–  Eachvertexinthegraphisarandomvariableinadistribution.

•  EachvariableVisgivenanequationthatdeterministicallydefinesthevalueofVasafunctionofits“parents”.–  Suchequationsarepostulatedtobestructural,inthesensethatitfollowsthecause-effectdirection.

TheOperationalMeaning

•  This“DAGwithequations”iscausalinthesensethatitmustencodetheeffectsofaperfectintervention.

Rain Barometer

URainUBarometer

Rain=fR(URain)Barometer=fB(Rain,UBarometer)

Interventions

•  Anotherprimitive.Itisa“overriding”operator,setsavariabletoafixedvalueofinterest.Lowercasehererepresentsconstants.

r Barometer

URainUBarometer

Rain=rBarometer=fB(r,UBarometer)

Interventions

•  Itisthenotionofinterventionthatleadstotheasymmetricnatureofcausality.

Rain b

URainUBarometer

Rain=fR(URain)Barometer=b

Interventions

•  Itisthenotionofinterventionthatexplainswhy“correlationisnotcausation”.

Movingto

Florida

Dyingofoldage

Survivingoldage

yes Dyingofoldage

Survivingoldage

Notation:the“do”Operator

•  Wemustexpresshow“Dyingofoldage”varieswith“MovingtoFlorida”inbothcases.

•  Traditionally,conditionalprobabilitiescanbeusedforthat.Butnoticethat,inourexample,istrueintheobservationalcase(nointervention),butfalseintheinterventionalcase.

P(Dyingofoldage=True|MovingtoFlorida=True)≠P(Dyingofoldage=True|MovingtoFlorida=False)

Notation:the“do”Operator

•  InPearl’scalculus,thisisdistinguishedbyusingthe“do”operatortoindicateaninterventionasopposedtoanobservation.

P(Dyingofoldage=True|MovingtoFlorida=True)≠P(Dyingofoldage=True|MovingtoFlorida=False)

P(Dyingofoldage=True|do(MovingtoFlorida=True))=P(Dyingofoldage=True|do(MovingtoFlorida=False))

AveragesVs.Individuals

•  Thistypeofnotationcanbeusedtoexpresswhetheradrugiseffectiveornot,averagingoverapopulation,usingarandomizedcontrolledtrial:

•  Itdoesnotmakeanyclaims,however,onwhetherthereisabalanceofpositive/negativecasesthatcancelout.

P(Healthy=True|do(Treatment=Drug))?=P(Healthy=True|do(Treatment=Placebo))

Notation:CounterfactualIndices

•  Meanttocaptureindividual-levelvariability.

•  ForVjavariableinthesystem,andViavariablebeingintervenedatvaluev,weuseVj(v)asthecounterfactualvalueofVj,hadVibeingsettov.

•  Contextwilltelluswhichvariablethevalue“v”refersto.

Example

•  Notice:itiscommontorepresentVj(v)asjustVjifViisnota(directorindirect)causeofVj.

r Barometer(r)

Urain(r) Ubarometer(r)

r Barometer(r)

Urain Ubarometer

“OtherThingsBeingEqual”

•  Thatis,– Acounterfactualvaluereplacesthecauseofinterest

– Thecounterfactualvaluepropagates“downstream”thecausalgraphviathestructuralequations

– Everythingelseremainsthesame(“otherthingsbeingequal”),i.e.,thenon-descendantsofthemanipulatedvariable.

MultipleWorlds•  Acounterfactualisjustadifferent“version”ofthesameindividual.All“versions”co-existinonebigjointdistribution.

Rain Barometer

URainUBarometer

r Barometer(r)

r’ Barometer(r’)

Workflow




BacktoCounterfactualFairness

•  Thelawofcounterfactualpropagationmeansthat,ifwewanttoensureitissufficient(andnecessary,ingeneral)toincludeonlythenon-descendantsofAinthedefinitionofthepredictor.

P (Y (a) = y | A = a,X = x) = P (Y (a0) = y | A = a,X = x)<latexit sha1_base64="ikA4UWRxXdG1jbKtP7ZSuE5mo/g=">AAACL3icdVDLSgMxFM34rPU16tJNsIgVpMyIoBuhKojLCvYhnaHcSdM2NPMgyYhD7R+58Ve6EVHErX9hpi2orR4IOZxzLsk9XsSZVJb1YszMzs0vLGaWsssrq2vr5sZmRYaxILRMQh6KmgeSchbQsmKK01okKPgep1Wve5H61TsqJAuDG5VE1PWhHbAWI6C01DAvS3mnAwrf5mEfn+LEwQ8OPtMMDnBNX/ep+p3Z+yfUMHNWwRoCTxN7THJojFLDHDjNkMQ+DRThIGXdtiLl9kAoRjjtZ51Y0ghIF9q0rmkAPpVub7hvH+9qpYlbodAnUHio/pzogS9l4ns66YPqyEkvFf/y6rFqnbg9FkSxogEZPdSKOVYhTsvDTSYoUTzRBIhg+q+YdEAAUbrirC7Bnlx5mlQOC7ZVsK+PcsXzcR0ZtI12UB7Z6BgV0RUqoTIi6BEN0Ct6M56MZ+Pd+BhFZ4zxzBb6BePzC/vYoRk=</latexit><latexit sha1_base64="ikA4UWRxXdG1jbKtP7ZSuE5mo/g=">AAACL3icdVDLSgMxFM34rPU16tJNsIgVpMyIoBuhKojLCvYhnaHcSdM2NPMgyYhD7R+58Ve6EVHErX9hpi2orR4IOZxzLsk9XsSZVJb1YszMzs0vLGaWsssrq2vr5sZmRYaxILRMQh6KmgeSchbQsmKK01okKPgep1Wve5H61TsqJAuDG5VE1PWhHbAWI6C01DAvS3mnAwrf5mEfn+LEwQ8OPtMMDnBNX/ep+p3Z+yfUMHNWwRoCTxN7THJojFLDHDjNkMQ+DRThIGXdtiLl9kAoRjjtZ51Y0ghIF9q0rmkAPpVub7hvH+9qpYlbodAnUHio/pzogS9l4ns66YPqyEkvFf/y6rFqnbg9FkSxogEZPdSKOVYhTsvDTSYoUTzRBIhg+q+YdEAAUbrirC7Bnlx5mlQOC7ZVsK+PcsXzcR0ZtI12UB7Z6BgV0RUqoTIi6BEN0Ct6M56MZ+Pd+BhFZ4zxzBb6BePzC/vYoRk=</latexit><latexit sha1_base64="ikA4UWRxXdG1jbKtP7ZSuE5mo/g=">AAACL3icdVDLSgMxFM34rPU16tJNsIgVpMyIoBuhKojLCvYhnaHcSdM2NPMgyYhD7R+58Ve6EVHErX9hpi2orR4IOZxzLsk9XsSZVJb1YszMzs0vLGaWsssrq2vr5sZmRYaxILRMQh6KmgeSchbQsmKK01okKPgep1Wve5H61TsqJAuDG5VE1PWhHbAWI6C01DAvS3mnAwrf5mEfn+LEwQ8OPtMMDnBNX/ep+p3Z+yfUMHNWwRoCTxN7THJojFLDHDjNkMQ+DRThIGXdtiLl9kAoRjjtZ51Y0ghIF9q0rmkAPpVub7hvH+9qpYlbodAnUHio/pzogS9l4ns66YPqyEkvFf/y6rFqnbg9FkSxogEZPdSKOVYhTsvDTSYoUTzRBIhg+q+YdEAAUbrirC7Bnlx5mlQOC7ZVsK+PcsXzcR0ZtI12UB7Z6BgV0RUqoTIi6BEN0Ct6M56MZ+Pd+BhFZ4zxzBb6BePzC/vYoRk=</latexit><latexit sha1_base64="ikA4UWRxXdG1jbKtP7ZSuE5mo/g=">AAACL3icdVDLSgMxFM34rPU16tJNsIgVpMyIoBuhKojLCvYhnaHcSdM2NPMgyYhD7R+58Ve6EVHErX9hpi2orR4IOZxzLsk9XsSZVJb1YszMzs0vLGaWsssrq2vr5sZmRYaxILRMQh6KmgeSchbQsmKK01okKPgep1Wve5H61TsqJAuDG5VE1PWhHbAWI6C01DAvS3mnAwrf5mEfn+LEwQ8OPtMMDnBNX/ep+p3Z+yfUMHNWwRoCTxN7THJojFLDHDjNkMQ+DRThIGXdtiLl9kAoRjjtZ51Y0ghIF9q0rmkAPpVub7hvH+9qpYlbodAnUHio/pzogS9l4ns66YPqyEkvFf/y6rFqnbg9FkSxogEZPdSKOVYhTsvDTSYoUTzRBIhg+q+YdEAAUbrirC7Bnlx5mlQOC7ZVsK+PcsXzcR0ZtI12UB7Z6BgV0RUqoTIi6BEN0Ct6M56MZ+Pd+BhFZ4zxzBb6BePzC/vYoRk=</latexit>

•  A,Employedcannotbeused.PrejudicedandQualificationscan.

•  IfitisjudgedthatPrejudicedcannotbeused,itshouldbelabelledasaprotectedattributed.

Examples

•  A,Xcannotbeused.Ucan.

HowtoExtractUnobservedVariables?

•  Usethe“factual”distributiontogetadistributionovertheunobservedvariablesbystandardprobabilisticconditioning.

•  MonteCarlodataaugmentationapproach:replaceeachdatapointinyourtrainingsamplebyasetoftrainingpointswiththeunobservedvariablesbeingfilledbyaMonteCarlosample.

P(Unobserved|Observed)

Workflow




Algorithm

Interpretation

•  ExtractcausesofYwhicharenotmediatorsbetweenAandY.

•  Findthe“bestapproximation”toYwithinthespaceoffunctionsthatexcludesuchmediators.

•  EvenifYis“unfair”(Aisacauseofit),byconstructionthepredictorwillbecounterfactuallyfair.

Challenges

•  Counterfactualfairnessclarifiesthatalgorithmicfairnessingeneralisnotexplicitlymodelinghowthewordbecomesfairerwithfairpredictions.–  Evenifourdecisionofgivingaloanisfair,itdoesn’tmeanthatinaggregatetheprobabilityofapersonofaparticulardemographicgroupwon’thavedifficultiesinrepayingit(AstillcausesY).

•  Thedelayedimpactoffairpredictionsisalsoaresearchtopic–  seeLiuetal.,https://arxiv.org/pdf/1803.04383.pdf

Workflow




SomeWordsofCaution•  Structuralequationsuseunobservedvariables.

•  Itiscommonthatsomeofthesevariablesare“default”choicesbasedonsomegenericmodelingassumptionsuchasadditiveerrors.

•  Natureandsocietycouldn’tcarelesswhetheryourmathematicallyconvenientwayofseparatingsignaland“noise”iselegantornot.

Output=Signal+Noise

SomeWordsofCaution

•  Thatis,thereareinfinitelymanystructuralequationsVj=fj(Vi,Uj)compatiblewithP(Vi|Vj)andP(Vi|do(Vj)).

•  Signalvs.noisemustbedeterminedbyreal-worldassumptions(“simplicity”assumptions,oftheOckham’srazortype,canbesometimesadequateaslongascaveatsareadvertised).

SomeWordsofCaution

•  Bynow,thereareseveralgoodpapersonhowtotacklefairnessbygeneratingunobservedvariableswhichareindependentofA,usingassortedmethods.

X

U

A“Thenoffyougotoplug-inUonamachinelearningalgorithm!”

However•  Therearepapersnotcausallymotivated,whichIfindof

difficultinterpretation.–  Remember:thereareinfinitelymanywaysofextractingU.

•  Therearepaperscausallymotivated,butwhichcommitthemselvestoadomain-freefamilyofstructuralequations.OKenough,butwhywouldyoudothat?–  Counterfactualfairnessemphasizesthatthecausalmodelingstepisseparatefromthepredictionlearningprocess.

•  Finally,dobewareofanypaperthatclaimstodoassumption-freeextractionof“causallatentfactors”.Thosearesellingyousnakeoil!

InterpretationofCounterfactuals•  Butwhatdoesitmeantosay“hadmyracebeendifferent”??–  First,makesuretounderstandthedifferencebetween“A”and“PerceptionofA”:thesecanleadtoconceptuallydifferentinterpretations,evenifthemodelstaysthesame.

– Withoutgoingindetails,ifthosecounterfactualsmakeyoufeeluneasy,justinterpretthemascomparingtwodifferentpeoplewhohappentomatchonthe“otherthingsbeingequal”factors.•  Thisisalsorelatedtofairnessthroughawareness(Dworketal.,2011,https://arxiv.org/abs/1104.3913)

Non-CounterfactualCausalModels

•  Contrarytofolkknowledge,causalitydoesnotrequirecounterfactuals:the“do”operatorisanwayofcomparingtreatmentswithoutcomparingindividuals.

•  However,iffeaturesXareaffectedbyA,theningeneralthereisnoindividualwhere

•  IffeaturesXarenotaffectedbyA,thenwecanshowwedonotneedtoexplicitlymodelstructuralequationsanyway!

P(Y=y|X=x,do(A=a))=P(Y=y|X=x,do(A=a’))

TheUpside•  Becausestructuralcausalmodelsrelyonunobservedvariables,atleasttheycanbepartiallyfalsifiedbyeventuallymeasuringsomeofthosevariables.

•  Justkeepinmind:–  whileitispreposteroustosayyouhave“the”causalmodelofasocialprocess,youshould(must?)beabletoexplainyourassumptionstoaregulatororacustomer.

–  Havingpassedtestableimplications,theremainingcomponentsofacounterfactualmodelshouldbeunderstoodasconjecturesformulatedaccordingtothebestofourknowledge.Suchmodelsshouldalwaysbedeemedprovisionalandpronetomodifications.

Illustration

•  TheLawSchoolAdmissionCouncilconductedasurveyacross163lawschoolsintheUnitedStates–  Itcontainsinformationon21,790lawstudentssuchastheirentranceexamscores(LSAT),theirgrade-pointaverage(GPA)collectedpriortolawschool,andtheirfirstyearaveragegrade(FYA).

•  Task:predictifanapplicantwillhaveahighFYA– Exampleofdecisionproblem:makeanoffer

Setup

•  Iwillpresentsomesimplecausalmodelsforthisdomain,whichbynomeansIintendtosellaswell-thoughtmodels.Theirpurposeisforillustration.

•  Wewillfitrealdatatoamodel,thengeneratesyntheticcounterfactualsoutofit.Thepointistoquantifytowhatextentacausally-obliviousmethodviolatescounterfactualfairness.

TwoModels

“FairK” “FairAdd”

PredictiveError(RealData)

•  Comparisonagainst“Full”(linearmodelwithallvariables)and“Unaware”(linearmodelwithoutraceandgender,buttheothertwopredictors)– Evaluationbyrootmeansquarederror

FairnessViolations(SimulatedCounterfactuals)

Extension:UsingMultipleModels

•  Wejustsawtwodifferentcounterfactualmodelsthatgivedifferentpredictionsdespitebeingundistinguishablegiventhesamedata.

•  ThisisOKassuminglittledifferencebetweenmodels,butwemayhavecompetingtheorieswithsomesizeabledifference.Wewouldliketobe“approximatelycounterfactuallyfair”toallofthem.

(ε,δ)-CounterfactualFairness•  Thefollowingconstraintprovidesarelaxationofcounterfactualfairness:

•  Theideaistosimultaneouslysatisfysuchconstraintsaccordingtodifferentcounterfactualmodels.–  Itisnothardtoshowthatthisproblemingeneralhasnosolutionifε=0,hencetheneedforanapproximateversion.

P (|Y (a)� Y (a0)| ✏ | A = a,X = a) � 1� �<latexit sha1_base64="6ftP5coO2gomwvZHxhd+kDhDw4c=">AAACM3icbVBdaxNBFJ1ta61R29Q++nIxiAnYsFsK+iL040X6FMG0kWwIdyc3ydDZ2XXmbiEk/U996R/xQSg+KMVX/4OzaQra9sAwh3PuYeaeJNfKcRheBUvLK49WH689qTx99nx9o7r54thlhZXUlpnObCdBR1oZarNiTZ3cEqaJppPk9LD0T87IOpWZzzzJqZfiyKihkshe6lePWvVZPEaGL3VswDbc8jeNGcSavkJMuVM6MzHMYtiHD4BvoVNeDYhH3o/K0IA0Y79aC5vhHHCfRAtSEwu0+tVv8SCTRUqGpUbnulGYc2+KlpXUdF6JC0c5ylMcUddTgym53nS+8zm89soAhpn1xzDM1X8TU0ydm6SJn0yRx+6uV4oPed2Ch+97U2XygsnIm4eGhQbOoCwQBsqSZD3xBKVV/q8gx2hRsq+54kuI7q58nxzvNKOwGX3are0dLOpYEy/FK1EXkXgn9sRH0RJtIcWF+C5+il/BZfAjuA5+34wuBYvMlvgPwZ+/koulag==</latexit><latexit sha1_base64="6ftP5coO2gomwvZHxhd+kDhDw4c=">AAACM3icbVBdaxNBFJ1ta61R29Q++nIxiAnYsFsK+iL040X6FMG0kWwIdyc3ydDZ2XXmbiEk/U996R/xQSg+KMVX/4OzaQra9sAwh3PuYeaeJNfKcRheBUvLK49WH689qTx99nx9o7r54thlhZXUlpnObCdBR1oZarNiTZ3cEqaJppPk9LD0T87IOpWZzzzJqZfiyKihkshe6lePWvVZPEaGL3VswDbc8jeNGcSavkJMuVM6MzHMYtiHD4BvoVNeDYhH3o/K0IA0Y79aC5vhHHCfRAtSEwu0+tVv8SCTRUqGpUbnulGYc2+KlpXUdF6JC0c5ylMcUddTgym53nS+8zm89soAhpn1xzDM1X8TU0ydm6SJn0yRx+6uV4oPed2Ch+97U2XygsnIm4eGhQbOoCwQBsqSZD3xBKVV/q8gx2hRsq+54kuI7q58nxzvNKOwGX3are0dLOpYEy/FK1EXkXgn9sRH0RJtIcWF+C5+il/BZfAjuA5+34wuBYvMlvgPwZ+/koulag==</latexit><latexit sha1_base64="6ftP5coO2gomwvZHxhd+kDhDw4c=">AAACM3icbVBdaxNBFJ1ta61R29Q++nIxiAnYsFsK+iL040X6FMG0kWwIdyc3ydDZ2XXmbiEk/U996R/xQSg+KMVX/4OzaQra9sAwh3PuYeaeJNfKcRheBUvLK49WH689qTx99nx9o7r54thlhZXUlpnObCdBR1oZarNiTZ3cEqaJppPk9LD0T87IOpWZzzzJqZfiyKihkshe6lePWvVZPEaGL3VswDbc8jeNGcSavkJMuVM6MzHMYtiHD4BvoVNeDYhH3o/K0IA0Y79aC5vhHHCfRAtSEwu0+tVv8SCTRUqGpUbnulGYc2+KlpXUdF6JC0c5ylMcUddTgym53nS+8zm89soAhpn1xzDM1X8TU0ydm6SJn0yRx+6uV4oPed2Ch+97U2XygsnIm4eGhQbOoCwQBsqSZD3xBKVV/q8gx2hRsq+54kuI7q58nxzvNKOwGX3are0dLOpYEy/FK1EXkXgn9sRH0RJtIcWF+C5+il/BZfAjuA5+34wuBYvMlvgPwZ+/koulag==</latexit><latexit sha1_base64="6ftP5coO2gomwvZHxhd+kDhDw4c=">AAACM3icbVBdaxNBFJ1ta61R29Q++nIxiAnYsFsK+iL040X6FMG0kWwIdyc3ydDZ2XXmbiEk/U996R/xQSg+KMVX/4OzaQra9sAwh3PuYeaeJNfKcRheBUvLK49WH689qTx99nx9o7r54thlhZXUlpnObCdBR1oZarNiTZ3cEqaJppPk9LD0T87IOpWZzzzJqZfiyKihkshe6lePWvVZPEaGL3VswDbc8jeNGcSavkJMuVM6MzHMYtiHD4BvoVNeDYhH3o/K0IA0Y79aC5vhHHCfRAtSEwu0+tVv8SCTRUqGpUbnulGYc2+KlpXUdF6JC0c5ylMcUddTgym53nS+8zm89soAhpn1xzDM1X8TU0ydm6SJn0yRx+6uV4oPed2Ch+97U2XygsnIm4eGhQbOoCwQBsqSZD3xBKVV/q8gx2hRsq+54kuI7q58nxzvNKOwGX3are0dLOpYEy/FK1EXkXgn9sRH0RJtIcWF+C5+il/BZfAjuA5+34wuBYvMlvgPwZ+/koulag==</latexit>

LawSchoolRevisited

COMPAS

ADifferentDirection:Interventions•  Sofar,wehavesolelydiscussedthecreationofpredictors.

•  Ideally,wewouldliketodestroythepathwaysbetweenAandY,theoutcomeofinterest.

•  Thisisingeneralnotpossible.Butlet’sassumewehaveaninterventionwiththeabilityofchangingthecontributionofAtoY.Howisrelatedtocounterfactualfairness?

ImperfectInterventionsandInterference

•  Wewillassumetwogeneralizationsoftheconceptofinterventionusedsofar.

•  Aninterventionisrepresentedgenericallyasasetof(action)variables,whichhereIwilldenoteasZ.– WecandefineZ=0asthe“noaction”choice!– Z≠0justmeansthatoneormorestructuralequationswillchange,notnecessarilytoaconstant(“imperfect”,or“soft”intervention).

Ideally

•  Havingavailablesome“Z=z”whichcompletelyoverridesthestructuralequationforYtonotdependonanythingthatstartsonA.

Y

A

Z

InReality

•  Nosuchaninterventionistypicallyavailable.•  Andthisisnotapredictionproblemanymore.WhathappenstoY?

•  Setup:– AssumeYisencodedsothathighvaluesaregood.– Modelallowsforinterference:thatis,treatmentZigiventopersonimightaffectpersonj.

– Howisthisrelatedtocounterfactualfairness?

OptimizationProblemandConstraints

E[Yi(ai, z) | Ai = ai, Xi = xi]� E[Yi(a0, z) | Ai = ai, Xi = xi]| {z }

Gia0

< ⌧

<latexit sha1_base64="4c9mlclgUDbW9Ia5Ysx9IkEqukU=">AAACe3icbVFdaxNBFJ1dv2r8ivooyMUgidKGXVHsg0KriD5WMG0kuwx3J3fbobOzy8ysNK77J/xpffOf+CI42eRB0x4YOJxzD3fuvVmlpHVR9CsIr1y9dv3G1s3erdt37t7r339waMvaCJqIUpVmmqElJTVNnHSKppUhLDJFR9np+6V/9I2MlaX+4hYVpQUea5lLgc5LvP8zqfWcTGZQUNNLCnQnWQYfZl+5HCGX27CScvj+LIEfkMA+l/AWOmva0TMuU9iBzexwI3pJsou2vPnIG4nDtoU3kDiseX8QjaMOcJHEazJgaxzw/nkyL0VdkHZCobWzOKpc2qBxUihqe0ltqUJxisc081RjQTZtut218NQrc8hL45920Kn/JhosrF0Uma9cjmM3vaV4mTerXb6bNlJXtSMtVo3yWoErYXkImEtDwqmFJyiM9H8FcYL+Ds6fq+eXEG+OfJEcvhjH0Tj+/HKw9269ji32iD1hIxaz12yPfWIHbMIE+x08DobBKPgTDsLn4faqNAzWmYfsP4Sv/gIWTboB</latexit><latexit sha1_base64="4c9mlclgUDbW9Ia5Ysx9IkEqukU=">AAACe3icbVFdaxNBFJ1dv2r8ivooyMUgidKGXVHsg0KriD5WMG0kuwx3J3fbobOzy8ysNK77J/xpffOf+CI42eRB0x4YOJxzD3fuvVmlpHVR9CsIr1y9dv3G1s3erdt37t7r339waMvaCJqIUpVmmqElJTVNnHSKppUhLDJFR9np+6V/9I2MlaX+4hYVpQUea5lLgc5LvP8zqfWcTGZQUNNLCnQnWQYfZl+5HCGX27CScvj+LIEfkMA+l/AWOmva0TMuU9iBzexwI3pJsou2vPnIG4nDtoU3kDiseX8QjaMOcJHEazJgaxzw/nkyL0VdkHZCobWzOKpc2qBxUihqe0ltqUJxisc081RjQTZtut218NQrc8hL45920Kn/JhosrF0Uma9cjmM3vaV4mTerXb6bNlJXtSMtVo3yWoErYXkImEtDwqmFJyiM9H8FcYL+Ds6fq+eXEG+OfJEcvhjH0Tj+/HKw9269ji32iD1hIxaz12yPfWIHbMIE+x08DobBKPgTDsLn4faqNAzWmYfsP4Sv/gIWTboB</latexit><latexit sha1_base64="4c9mlclgUDbW9Ia5Ysx9IkEqukU=">AAACe3icbVFdaxNBFJ1dv2r8ivooyMUgidKGXVHsg0KriD5WMG0kuwx3J3fbobOzy8ysNK77J/xpffOf+CI42eRB0x4YOJxzD3fuvVmlpHVR9CsIr1y9dv3G1s3erdt37t7r339waMvaCJqIUpVmmqElJTVNnHSKppUhLDJFR9np+6V/9I2MlaX+4hYVpQUea5lLgc5LvP8zqfWcTGZQUNNLCnQnWQYfZl+5HCGX27CScvj+LIEfkMA+l/AWOmva0TMuU9iBzexwI3pJsou2vPnIG4nDtoU3kDiseX8QjaMOcJHEazJgaxzw/nkyL0VdkHZCobWzOKpc2qBxUihqe0ltqUJxisc081RjQTZtut218NQrc8hL45920Kn/JhosrF0Uma9cjmM3vaV4mTerXb6bNlJXtSMtVo3yWoErYXkImEtDwqmFJyiM9H8FcYL+Ds6fq+eXEG+OfJEcvhjH0Tj+/HKw9269ji32iD1hIxaz12yPfWIHbMIE+x08DobBKPgTDsLn4faqNAzWmYfsP4Sv/gIWTboB</latexit><latexit sha1_base64="4c9mlclgUDbW9Ia5Ysx9IkEqukU=">AAACe3icbVFdaxNBFJ1dv2r8ivooyMUgidKGXVHsg0KriD5WMG0kuwx3J3fbobOzy8ysNK77J/xpffOf+CI42eRB0x4YOJxzD3fuvVmlpHVR9CsIr1y9dv3G1s3erdt37t7r339waMvaCJqIUpVmmqElJTVNnHSKppUhLDJFR9np+6V/9I2MlaX+4hYVpQUea5lLgc5LvP8zqfWcTGZQUNNLCnQnWQYfZl+5HCGX27CScvj+LIEfkMA+l/AWOmva0TMuU9iBzexwI3pJsou2vPnIG4nDtoU3kDiseX8QjaMOcJHEazJgaxzw/nkyL0VdkHZCobWzOKpc2qBxUihqe0ltqUJxisc081RjQTZtut218NQrc8hL45920Kn/JhosrF0Uma9cjmM3vaV4mTerXb6bNlJXtSMtVo3yWoErYXkImEtDwqmFJyiM9H8FcYL+Ds6fq+eXEG+OfJEcvhjH0Tj+/HKw9269ji32iD1hIxaz12yPfWIHbMIE+x08DobBKPgTDsLn4faqNAzWmYfsP4Sv/gIWTboB</latexit>

maxz1,...,zn

nX

i=1

E[Yi(z) | Ai = ai, Xi = xi]

s.t.,nX

i=1

zi B

Gia0 ⌧ 8a0 2 A, i 2 1, . . . , n,<latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="Z/0GUlhD2yyTXjoOAb1lGkJlzSA=">AAAC0HicbVLLbtQwFHXCq4QCA1s2V1TQIkWjpBsQo0oUhGBZJKYdNA7WjcdprTpOGjuoMyFsWIAQX8eO3+ALcCYjAVOuZOn4nPu201JJY6Pop+dfunzl6rWN68GNzZu3bg/ubB6aoq64GPNCFdUkRSOU1GJspVViUlYC81SJo/T0RacffRCVkYV+a+elSHI81jKTHK2j2OAXzfGcNQsWh0DVrLAmhAXTLVBT56yRe3H7Xj+kI3B+9iRN4eX0HZM7/S2DxSMKHynsMwl7gEyGMFnCcyYTlw9ToRpx9rQobUspDczQDl2d0XryhQuiSpzBc6C60HWeigqcf6e9co643fY6tVh38aOuo6yoUCnAbaBS9w1yVLAfglwycbiaSId/0gYBG2xFw2hpcBHEK7BFVnbABj/orOB1LrTlCo2ZxlFpkwYrK7kSbUBrI0rkp3gspg5qzIVJmuXbtPDAMTNwrbqjLSzZvyMazI2Z56nz7AYw61pH/k+b1jZ7kjRSl7UVmveFslqBLaB7aJjJSnCr5g4gr6TrFfgJVsit+w7dEuL1kS+Cw91hHA3jNxHZIPfIfbJDYvKYPCOvyQEZE+5NvE/eF++r n/if/W/9unxvtbe75B/zv/8GcCncCQ==</latexit><latexit sha1_base64="Z/0GUlhD2yyTXjoOAb1lGkJlzSA=">AAAC0HicbVLLbtQwFHXCq4QCA1s2V1TQIkWjpBsQo0oUhGBZJKYdNA7WjcdprTpOGjuoMyFsWIAQX8eO3+ALcCYjAVOuZOn4nPu201JJY6Pop+dfunzl6rWN68GNzZu3bg/ubB6aoq64GPNCFdUkRSOU1GJspVViUlYC81SJo/T0RacffRCVkYV+a+elSHI81jKTHK2j2OAXzfGcNQsWh0DVrLAmhAXTLVBT56yRe3H7Xj+kI3B+9iRN4eX0HZM7/S2DxSMKHynsMwl7gEyGMFnCcyYTlw9ToRpx9rQobUspDczQDl2d0XryhQuiSpzBc6C60HWeigqcf6e9co643fY6tVh38aOuo6yoUCnAbaBS9w1yVLAfglwycbiaSId/0gYBG2xFw2hpcBHEK7BFVnbABj/orOB1LrTlCo2ZxlFpkwYrK7kSbUBrI0rkp3gspg5qzIVJmuXbtPDAMTNwrbqjLSzZvyMazI2Z56nz7AYw61pH/k+b1jZ7kjRSl7UVmveFslqBLaB7aJjJSnCr5g4gr6TrFfgJVsit+w7dEuL1kS+Cw91hHA3jNxHZIPfIfbJDYvKYPCOvyQEZE+5NvE/eF++r n/if/W/9unxvtbe75B/zv/8GcCncCQ==</latexit><latexit sha1_base64="0TQT7/ceH+4OXOofio+LZ2mzcLc=">AAAC23icbVJNb9QwEHXCVwlfCxy5jFhBixStEi4gVpXaIgTHIrHtonWwJl5va9Vx0thB3Q3hwgGEuPLHuPE3+AU4m5WALSNZen7vzXjGdlooaWwU/fT8CxcvXb6ycTW4dv3GzVu923cOTF6VXIx4rvJynKIRSmoxstIqMS5KgVmqxGF68rzVD9+L0shcv7HzQiQZHmk5kxyto1jvF83wjNULFodA1TS3JoQF0w1QU2Wslttx804/pENwPnucpvBi8pbJrW43g8UjCh8o7DIJ24BMhjBewjMmE1cPU6FqcfosL2xDKQ3MwA7cOcP14guXRJU4hT2gOtdVlooSnL/VXjojbjadTi1Wbf6w7WiWl6gU4CZQqbsGOSrYDUEumThcTaTDP2WDgPX60SBaBpwH8Qr0ySr2We8Hnea8yoS2XKExkzgqbFJjaSVXogloZUSB/ASPxMRBjZkwSb18mwYeOGYKrlW3tIUl+3dGjZkx8yx1znYAs6615P+0SWVnT5Na6qKyQvPuoFmlwObQPjRMZSm4VXMHkJfS9Qr8GEvk1n2H9hLi9ZHPg4PHgzgaxK+j/s7e6jo2yD1yn2yRmDwhO+QV2Scj wr2x99H77H3xE/+T/9X/1ll9b5Vzl/wT/vff/3vdTA==</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit><latexit sha1_base64="wE8PNAxqRV0xhraKiAlRilriD8Q=">AAAC23icbVJdb9MwFHXCx0b5WIFHXq6oYEOKqgQhDVFN2oYQPA6JbkV1sG5cZ7PmOFnsoLUhvPAAQrzyx3jjb/ALcNpKQMeVLB2fc+71vbaTQkljw/Cn51+6fOXq2vq1zvUbN29tdG/fOTR5VXIx5LnKy1GCRiipxdBKq8SoKAVmiRJHyenzVj96L0ojc/3GTgsRZ3isZSo5Wkex7i+a4TmrZywKgKpJbk0AM6YboKbKWC13ouadfkgH4Hz2JEngxfgtk1uLXQqzRxQ+UNhjEnYAmQxgNIfnTMauHiZC1eLsWV7YhlLaMX3bd+cMVovPXBJV4gz2gepcV1kiSnD+VnvpjLjZLHRqsWrzB21HaV6iUoCbQKVeNMhRwV4Acs5EwXIiHfwp2+mwbi/sh/OAiyBagh5ZxgHr/qCTnFeZ0JYrNGYchYWNayyt5Eo0HVoZUSA/xWMxdlBjJkxcz9+mgQeOmYBr1S1tYc7+nVFjZsw0S5yzHcCsai35P21c2fRpXEtdVFZovjgorRTYHNqHhoksBbdq6gDyUrpegZ9gidy679BeQrQ68kVw+Lgfhf3o9ZPe7v7yOtbJPXKfbJGIbJNd8ooc kCHh3sj76H32vvix/8n/6n9bWH1vmXOX/BP+998Ayt1Q</latexit>

Mainfamilyofconstraints:

(asopposedto|E[Yi(ai, z) | Ai = ai, Xi = xi]� E[Yi(a

0, z) | Ai = ai, Xi = xi]| < ⌧<latexit sha1_base64="xWNizRs8+QU/6klSOqWWnZ6CX5Y=">AAACZnicbVFLS8NAGNzEd3zFinjw8mERFbQkIuhBwQeCxwpWK00Im+2mXbp5sLsRa9s/6c2zF3+GSdqDpg4sDDPfsPvN+glnUlnWp6bPzM7NLywuGcsrq2vr5kblScapILRBYh6Lpo8l5SyiDcUUp81EUBz6nD77vdvcf36lQrI4elT9hLoh7kQsYASrTPLM0dAJser6Pty1Xjx2gD12BGMpgPdDB4bgwLXH4BIKq1nQN4+5cAxGKbtfiv6TLKJDuABH4dTwzKpVswrANLEnpIomqHvmh9OOSRrSSBGOpWzZVqLcARaKEU5HhpNKmmDSwx3aymiEQyrdQVHTCPYypQ1BLLITKSjU34kBDqXsh342mW8hy14u/ue1UhWcuwMWJamiERlfFKQcVAx559BmghLF+xnBRLDsrUC6WGCisp/JS7DLK0+Tp5OabdXsh9Pq1c2kjkW0g3bRAbLRGbpC96iOGoigL21Jq2ib2re+pm/p2+NRXZtkNtEf6PADY1+w0g==</latexit><latexit sha1_base64="xWNizRs8+QU/6klSOqWWnZ6CX5Y=">AAACZnicbVFLS8NAGNzEd3zFinjw8mERFbQkIuhBwQeCxwpWK00Im+2mXbp5sLsRa9s/6c2zF3+GSdqDpg4sDDPfsPvN+glnUlnWp6bPzM7NLywuGcsrq2vr5kblScapILRBYh6Lpo8l5SyiDcUUp81EUBz6nD77vdvcf36lQrI4elT9hLoh7kQsYASrTPLM0dAJser6Pty1Xjx2gD12BGMpgPdDB4bgwLXH4BIKq1nQN4+5cAxGKbtfiv6TLKJDuABH4dTwzKpVswrANLEnpIomqHvmh9OOSRrSSBGOpWzZVqLcARaKEU5HhpNKmmDSwx3aymiEQyrdQVHTCPYypQ1BLLITKSjU34kBDqXsh342mW8hy14u/ue1UhWcuwMWJamiERlfFKQcVAx559BmghLF+xnBRLDsrUC6WGCisp/JS7DLK0+Tp5OabdXsh9Pq1c2kjkW0g3bRAbLRGbpC96iOGoigL21Jq2ib2re+pm/p2+NRXZtkNtEf6PADY1+w0g==</latexit><latexit sha1_base64="xWNizRs8+QU/6klSOqWWnZ6CX5Y=">AAACZnicbVFLS8NAGNzEd3zFinjw8mERFbQkIuhBwQeCxwpWK00Im+2mXbp5sLsRa9s/6c2zF3+GSdqDpg4sDDPfsPvN+glnUlnWp6bPzM7NLywuGcsrq2vr5kblScapILRBYh6Lpo8l5SyiDcUUp81EUBz6nD77vdvcf36lQrI4elT9hLoh7kQsYASrTPLM0dAJser6Pty1Xjx2gD12BGMpgPdDB4bgwLXH4BIKq1nQN4+5cAxGKbtfiv6TLKJDuABH4dTwzKpVswrANLEnpIomqHvmh9OOSRrSSBGOpWzZVqLcARaKEU5HhpNKmmDSwx3aymiEQyrdQVHTCPYypQ1BLLITKSjU34kBDqXsh342mW8hy14u/ue1UhWcuwMWJamiERlfFKQcVAx559BmghLF+xnBRLDsrUC6WGCisp/JS7DLK0+Tp5OabdXsh9Pq1c2kjkW0g3bRAbLRGbpC96iOGoigL21Jq2ib2re+pm/p2+NRXZtkNtEf6PADY1+w0g==</latexit><latexit sha1_base64="xWNizRs8+QU/6klSOqWWnZ6CX5Y=">AAACZnicbVFLS8NAGNzEd3zFinjw8mERFbQkIuhBwQeCxwpWK00Im+2mXbp5sLsRa9s/6c2zF3+GSdqDpg4sDDPfsPvN+glnUlnWp6bPzM7NLywuGcsrq2vr5kblScapILRBYh6Lpo8l5SyiDcUUp81EUBz6nD77vdvcf36lQrI4elT9hLoh7kQsYASrTPLM0dAJser6Pty1Xjx2gD12BGMpgPdDB4bgwLXH4BIKq1nQN4+5cAxGKbtfiv6TLKJDuABH4dTwzKpVswrANLEnpIomqHvmh9OOSRrSSBGOpWzZVqLcARaKEU5HhpNKmmDSwx3aymiEQyrdQVHTCPYypQ1BLLITKSjU34kBDqXsh342mW8hy14u/ue1UhWcuwMWJamiERlfFKQcVAx559BmghLF+xnBRLDsrUC6WGCisp/JS7DLK0+Tp5OabdXsh9Pq1c2kjkW0g3bRAbLRGbpC96iOGoigL21Jq2ib2re+pm/p2+NRXZtkNtEf6PADY1+w0g==</latexit>

)

IntuitiveToyExample

•  ProtectedattributeAissuchthatAin{b,w},Xissomequantitativemeasureofprofessionalcompetence,andYisameasureofwealthin5years’time.

•  Zi=1meansindividualigetsasubsidytomovetoaneighborhoodwithbettertransportlinks.

IntuitiveToyExample

•  Supposestructuralequationis

•  Soiftherearetwoindividuals,oneoftypewandoneoftypeb,andZ1+Z2=1.

•  Withoutthefairnessconstraint,typewgetsthesubsidyeveniftypebhasupto50moreunitsofprofessionalability!

Yi = Xi + 100Zi + 50Zi ⇥ I(Ai = w) + Ui<latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="5cGLW4Oc6HfWRkuTbeluh14wMOc=">AAACGnicbVDLSgMxFL1TX7VWHd26CRahIpSMILpQUNzoroJ9aFuGTJq2oZkHSUYpQ3/Gjb/ixoVFBP/GzLQLbT2Q5HDuCffe40WCK43xt5VbWl5ZXcuvFzaKm1vb9k6xrsJYUlajoQhl0yOKCR6wmuZasGYkGfE9wRre8DqtN56YVDwM7vUoYh2f9APe45RoI7n2+YPL0QVqmvsIORg/ZuQke9ua+0yhtk/0wPOS23H5KjM/HxpLzeUF1y7hCs6AFokzIyWYoerak3Y3pLHPAk0FUarl4Eh3EiI1p4KNC+1YsYjQIemzlqEBMf07SbblGB0YpYt6oTQn0ChTf/9IiK/UyPeMM51YzddS8b9aK9a9s07CgyjWLKDTRr1YIB2iNDLU5ZJRLUaGECq5mRXRAZGEahNsGoIzv/IiqR9XHFxx7jDkYQ/2oQwOnMIl3EAVakDhBd7gAybWq/VufU7jylmz3HbhD6yvH9POnuo=</latexit><latexit sha1_base64="5cGLW4Oc6HfWRkuTbeluh14wMOc=">AAACGnicbVDLSgMxFL1TX7VWHd26CRahIpSMILpQUNzoroJ9aFuGTJq2oZkHSUYpQ3/Gjb/ixoVFBP/GzLQLbT2Q5HDuCffe40WCK43xt5VbWl5ZXcuvFzaKm1vb9k6xrsJYUlajoQhl0yOKCR6wmuZasGYkGfE9wRre8DqtN56YVDwM7vUoYh2f9APe45RoI7n2+YPL0QVqmvsIORg/ZuQke9ua+0yhtk/0wPOS23H5KjM/HxpLzeUF1y7hCs6AFokzIyWYoerak3Y3pLHPAk0FUarl4Eh3EiI1p4KNC+1YsYjQIemzlqEBMf07SbblGB0YpYt6oTQn0ChTf/9IiK/UyPeMM51YzddS8b9aK9a9s07CgyjWLKDTRr1YIB2iNDLU5ZJRLUaGECq5mRXRAZGEahNsGoIzv/IiqR9XHFxx7jDkYQ/2oQwOnMIl3EAVakDhBd7gAybWq/VufU7jylmz3HbhD6yvH9POnuo=</latexit><latexit sha1_base64="6VDkPHhJM2K4Gyz6mkhn0+38QmE=">AAACJXicbVDLSgMxFM34rPVVdekmWISKUDKC6EKh6kZ3FexDO8OQSTNtaOZBklHK0J9x46+4cWERwZW/YmY6C229kNzDuedy7z1uxJlUCH0Zc/MLi0vLhZXi6tr6xmZpa7spw1gQ2iAhD0XbxZJyFtCGYorTdiQo9l1OW+7gKq23HqmQLAzu1DCito97AfMYwUpTTuns3mHwHLb1fwhNhB4ycJxlSzGfSmj5WPVdN7kZVS4y8dOBljQcVnRKZVRFWcBZYOagDPKoO6Wx1Q1J7NNAEY6l7JgoUnaChWKE01HRiiWNMBngHu1oGGA9306yK0dwXzNd6IVCv0DBjP3dkWBfyqHvamW6sZyupeR/tU6svFM7YUEUKxqQySAv5lCFMLUMdpmgRPGhBpgIpneFpI8FJkobm5pgTp88C5pHVRNVzVtUrl3mdhTALtgDFWCCE1AD16AOGoCAZ/AK3sHYeDHejA/jcyKdM/KeHfAnjO8fsregdg==</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit><latexit sha1_base64="G6O0fZJi/VPIziL8S528M+CDGfg=">AAACJXicbVBNS8NAEN3Ur1q/oh69LBahIpREFD0oVL3orYJpq00Im+22Xbr5YHejlNA/48W/4sWDRQRP/hU3aQ7aOrA7jzdvmJnnRYwKaRhfWmFufmFxqbhcWlldW9/QN7caIow5JhYOWchbHhKE0YBYkkpGWhEnyPcYaXqDq7TefCRc0DC4k8OIOD7qBbRLMZKKcvWze5fCc9hS/wE0DeMhA8dZtiX1iYC2j2Tf85KbUeUiEz/tK4nl0pKrl42qkQWcBWYOyiCPuquP7U6IY58EEjMkRNs0IukkiEuKGRmV7FiQCOEB6pG2ggFS850ku3IE9xTTgd2QqxdImLG/OxLkCzH0PaVMNxbTtZT8r9aOZffUSWgQxZIEeDKoGzMoQ5haBjuUEyzZUAGEOVW7QtxHHGGpjE1NMKdPngWNw6ppVM3bo3LtMrejCHbALqgAE5yAGrgGdWABDJ7BK3gHY+1Fe9M+tM+JtKDlPdvgT2jfP7P3oHo=</latexit>

Considerations•  Theremightbenofeasiblesolutionifτissmallenough.

•  Itmightbethecasethatthe“counterfactualgap”ineachconstraintremainsconstantregardlessofZ.

•  Thesearefeaturesoftheintervention,notofthefairnessframework.Again,agoodinterventionisamatterofrealworlddesign,notofalgorithmdesign!

Illustration(PartiallySyntheticData)

•  NYCPublicSchools:interventionZistoprovidecalculusclassesinschools.

•  AttributeAiswhetherschoolhasawhitemajority.

•  OutcomeYisproportionofstudentstakingtheSAT/ACT.

•  Geographicalinterferenceisassumed.

Results

Conclusion

•  Iproposethatcausalmodelingshouldbeakeycomponentoffairnessconsiderations.

•  Fairnesshasmultiplefacets.Hereweconsideredpredictionandpolicy-makingunderinterference.

•  Muchmoreisrelevant:selectionbias,dynamicprediction/treatmentsetc.

•  Goodsoftwaredesigncouldhelpbuildingmassiveexperimentsintheinternet,forinstance.

References•  M.Kusner,C.Russell,J.LoftusandR.Silva(2017).“Counterfactual

Fairness”.NIPS2017.https://papers.nips.cc/paper/6995-counterfactual-fairness

•  C.Russell,M.Kusner,J.LoftusandR.Silva(2017).“WhenWorldsCollide:IntegratingDifferentCounterfactualAssumptionsinFairness”.NIPS2017.https://papers.nips.cc/paper/7220-when-worlds-collide-integrating-different-counterfactual-assumptions-in-fairness

•  J.Loftus,C.Russell,M.KusnerandR.Silva(2018).“CausalReasoningforAlgorithmicFairness”.https://arxiv.org/abs/1805.05859

•  M.Kusner,C.Russell,J.LoftusandR.Silva(2018).“CausalInterventionsforFairness”.TobeinarXivatanymoment.

ThankYou

Counterfactual Reasoning in Algorithmic Fairnessfairware.cs.umass.edu/slides/silva.pdf · Counterfactual Reasoning in Algorithmic Fairness Ricardo Silva University College London

Documents