Where are human subjects in big data research? The emerging ethics divide

J Metcalf, K Crawford - Big Data & Society, 2016 - journals.sagepub.com
There are growing discontinuities between the research practices of data science and established
tools of research ethics regulation. Some of the core commitments of existing research …

Algorithmic impact assessments and accountability: The co-construction of impacts

J Metcalf, E Moss, EA Watkins, R Singh… - Proceedings of the 2021 …, 2021 - dl.acm.org
Algorithmic impact assessments (AIAs) are an emergent form of accountability for organizations
that build and deploy automated decision-support systems. They are modeled after …

Owning ethics: Corporate logics, silicon valley, and the institutionalization of ethics

J Metcalf, E Moss - Social Research: An International Quarterly, 2019 - muse.jhu.edu
Jacob Metcalf jacob metcalf studies research ethics in data science, machine learning, and
artificial intelligence. He is a researcher at Data & Society, where he is co-project leader on …

The algorithmic imprint

U Ehsan, R Singh, J Metcalf, M Riedl - … of the 2022 ACM Conference on …, 2022 - dl.acm.org
When algorithmic harms emerge, a reasonable response is to stop using the algorithm to
resolve concerns related to fairness, accountability, transparency, and ethics (FATE). However, …

Assembling accountability: algorithmic impact assessment for the public interest

…, R Singh, MC Elish, J Metcalf - Available at SSRN …, 2021 - papers.ssrn.com
The Algorithmic Impact Assessment is a new concept for regulating algorithmic systems and
protecting the public interest. Assembling Accountability: Algorithmic Impact Assessment for …

Ten simple rules for responsible big data research

…, R Hollander, BA Koenig, J Metcalf… - PLoS computational …, 2017 - journals.plos.org
The use of big data research methods has grown tremendously over the past five years in
both academia and industry. As the size and complexity of available datasets has grown, so …

Taking algorithms to courts: A relational approach to algorithmic accountability

J Metcalf, R Singh, E Moss, E Tafesse… - Proceedings of the 2023 …, 2023 - dl.acm.org
In widely used sociological descriptions of how accountability is structured through
institutions, an “actor” (eg, the developer) is accountable to a “forum” (eg, regulatory agencies) …

Excavating awareness and power in data science: A manifesto for trustworthy pervasive data research

…, SA Gilbert, MJ Bietz, C Fiesler, J Metcalf… - Big Data & …, 2021 - journals.sagepub.com
Frequent public uproar over forms of data science that rely on information about people
demonstrates the challenges of defining and demonstrating trustworthy digital data research …

Auditing work: Exploring the New York City algorithmic bias audit regime

L Groves, J Metcalf, A Kennedy, B Vecchione… - The 2024 ACM …, 2024 - dl.acm.org
In July 2023, New York City (NYC) implemented the first attempt to create an algorithm auditing
regime for commercial machine-learning systems. Local Law 144 (LL 144), requires NYC…

Null Compliance: NYC Local Law 144 and the challenges of algorithm accountability

…, Comm 2450 Student Investigators, J Metcalf… - The 2024 ACM …, 2024 - dl.acm.org
In July 2023, New York City became the first jurisdiction globally to mandate bias audits for
commercial algorithmic systems, specifically for automated employment decisions systems (…