skip to main content
10.5555/3635637.3663112acmconferencesArticle/Chapter ViewAbstractPublication PagesaamasConference Proceedingsconference-collections
extended-abstract

A Reinforcement Learning Framework for Studying Group and Individual Fairness

Published: 06 May 2024 Publication History

Abstract

Reinforcement learning is a commonly used technique for optimising objectives in decision support systems for complex problem solving. When these systems affect individuals or groups, it is essential to reflect on fairness. As absolute fairness is in practice not achievable, we propose a framework which allows to balance distinct fairness notions along with the primary objective. To this end, we formulate group and individual fairness in sequential fairness notions. First, we present an extended Markov decision process, ƒMDP, that is explicitly aware of individuals and groups. Next, we formalise fairness notions in terms of this ƒMDP which allows us to evaluate the primary objective along with the fairness notions that are important to the user, taking a multi-objective reinforcement learning approach. To evaluate our framework, we consider two scenarios that require distinct aspects of the performance-fairness trade-off: job hiring and fraud detection. The objectives in job hiring are to compose strong teams, while providing equal treatment to similar individual applicants and to groups in society. The trade-off in fraud detection is the necessity of detecting fraudulent transactions, while distributing the burden for customers of checking transactions fairly. In this framework, we further explore the influence of distance metrics on individual fairness and highlight the impact of the history size on the fairness calculations and the obtainable fairness through exploration.

References

[1]
Vlaams Supercomputing Center. 2023. Hydra hardware. https://www.vscentrum. be https://www.vscentrum.be.
[2]
Jingdi Chen, Yimeng Wang, and Tian Lan. 2021. Bringing fairness to actor-critic reinforcement learning for network utility optimization. In IEEE INFOCOM 2021 - IEEE Conference on Computer Communications (Vancouver, BC, Canada). IEEE Press, Vancouver, BC, Canada, 1--10. https://doi.org/10.1109/INFOCOM42981. 2021.9488823
[3]
Alexandra Cimpean, Timothy Verstraeten, Lander Willem, Niel Hens, Ann Nowé, and Pieter Libin. 2023. Evaluating COVID-19 vaccine allocation policies using Bayesian m-top exploration. arXiv preprint arXiv:2301.12822 (2023), 26.
[4]
Ezekiel J. Emanuel, Govind Persad, Adam Kern, Allen Buchanan, Cécile Fabre, Daniel Halliday, Joseph Heath, Lisa Herzog, R. J. Leland, Ephrem T. Lemango, Florencia Luna, Matthew S. McCoy, Ole F. Norheim, Trygve Ottersen, G. Owen Schaefer, Kok-Chor Tan, Christopher Heath Wellman, Jonathan Wolff, and Henry S. Richardson. 2020. An ethical framework for global vaccine allocation. Science 369, 6509 (2020), 1309--1312. https://doi.org/10.1126/science.abe2803
[5]
Conor F. Hayes, Roxana R?dulescu, Eugenio Bargiacchi, Johan Källström, Matthew Macfarlane, Mathieu Reymond, Timothy Verstraeten, Luisa M. Zintgraf, Richard Dazeley, Fredrik Heintz, Enda Howley, Athirai A. Irissappane, Patrick Mannion, Ann Nowé, Gabriel Ramos, Marcello Restelli, Peter Vamplew, and Diederik M. Roijers. 2022. A practical guide to multi-objective reinforcement learning and planning. In AAMAS (2022/04/13), Vol. 36. 26.
[6]
Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. 2017. Fairness in Reinforcement Learning. In ICML (Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, Sydney, Australia, 1617--1626. https://proceedings.mlr.press/v70/jabbari17a.html
[7]
Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. 2016. Fairness in Learning: Classic and contextual bandits. Advances in Neural Information Processing Systems 29 (2016), 325--333. arXiv:1605.07139
[8]
Pieter J. K. Libin, Arno Moonens, Timothy Verstraeten, Fabian Perez-Sanjines, Niel Hens, Philippe Lemey, and Ann Nowé. 2021. Deep Reinforcement Learning for Large-Scale Epidemic Control. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track, Yuxiao Dong, Georgiana Ifrim, Dunja Mladenić, Craig Saunders, and Sofie Van Hoecke (Eds.). Springer International Publishing, Cham, 155--170.
[9]
Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2018. Delayed Impact of Fair Machine Learning. In ICML, Vol. 80. PMLR, Stockholm, Sweden, 3150--3158.
[10]
Weiwen Liu, Feng Liu, Ruiming Tang, Ben Liao, Guangyong Chen, and Pheng Ann Heng. 2020. Balancing Between Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning. Vol. 12084 LNAI. Springer International Publishing, Cham. 155--167 pages. https://doi.org/10.1007/978-3-030-47426-3_13 arXiv:2106.13386
[11]
Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. 2020. On the applicability of ML fairness notions., 32 pages. arXiv:2006.16745
[12]
Dennis Soemers, Ann Nowé, Tim Brys, Kurt Driessens, and Mark Winands. 2018. Adapting to Concept Drift in Credit Card Transaction Data Streams Using Contextual Bandits and Decision Trees. AAAI 32, 1 (2018), 7831--7836.
[13]
Mathieu Reymond, Eugenio Bargiacchi, and Ann Nowé. 2022. Pareto Conditioned Networks. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems (Virtual Event, New Zealand) (AAMAS '22). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1110--1118.
[14]
Manel Rodriguez-Soto, Maite Lopez-Sanchez, and Juan A Rodriguez-Aguilar. 2021. Guaranteeing the Learning of Ethical Behaviour through Multi-Objective Reinforcement Learning. ALA (2021), 9.
[15]
Candice Schumann, Samsara N. Counts, Jeffrey S. Foster, and John P. Dickerson. 2019. The Diverse Cohort Selection Problem. Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2 (2019), 601--609. arXiv:1709.03441
[16]
Candice Schumann, Jeffrey S. Foster, Nicholas Mattei, and John P. Dickerson. 2020. We need fairness and explainability in algorithmic hiring. Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2020-May, Aamas (2020), 1716--1720.
[17]
Umer Siddique, Paul Weng, and Matthieu Zimmer. 2020. Learning fair policies in multiobjective (Deep) reinforcement learning with Average and Discounted Rewards. ICML 119 (13-18 Jul 2020), 8864--8874. https://proceedings.mlr.press/ v119/siddique20a.html
[18]
STATBEL. 2023. Employment and unemployment. https: //statbel.fgov.be/en/themes/work-training/labour-market/employment-and-unemployment#figures
[19]
Richard S. Sutton, Andrew G. Barto, and et al. 2018. Reinforcement Learning: An Introduction. MIT Press. 526 pages.
[20]
Paul Weng. 2019. Fairness in reinforcement learning. CoRR abs/1907.10323 (2019), 5. arXiv:1907.10323
[21]
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning Fair Representations. In ICML (Proceedings of Machine Learning Research, Vol. 28), Sanjoy Dasgupta and David McAllester (Eds.). PMLR, Atlanta, Georgia, USA, 325--333.
[22]
Luisa M Zintgraf, Edgar A Lopez-Rojas, Diederik M Roijers, and Ann Nowé. 2017. MultiMAuS: a multi-modal authentication simulator for fraud detection research. In 29th European Modeling and Simulation Symp.(EMSS 2017). Curran Associates, Inc., 360--370.

Recommendations

Comments

Information & Contributors

Information

Published In

AAMAS '24: Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems
May 2024
2898 pages
ISBN:9798400704864

Sponsors

Publisher

International Foundation for Autonomous Agents and Multiagent Systems

Richland, SC

Publication History

Published: 06 May 2024

Check for updates

Author Tags

  1. automated decision support
  2. fairness framework
  3. reinforcement learning
  4. trustworthy ai

Qualifiers

  • Extended-abstract

Funding Sources

  • Fonds voor Wetenschappelijk Onderzoek (FWO)

Conference

AAMAS '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 20
    Total Downloads
  • Downloads (Last 12 months)20
  • Downloads (Last 6 weeks)3
Reflects downloads up to 22 Dec 2024

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media