Medical sciences have long since established an ethics code for experiments, to minimize
the risk of harm to subjects. Natural language processing (NLP) used to involve mostly …

Hate speech in the form of racist and sexist remarks are a common occurrence on social
media. For that reason, many social media services address the problem of identifying hate …

Without proper safeguards, large language models will readily follow malicious instructions
and generate toxic content. This risk motivates safety efforts such as red-teaming and large-…

Non-expert annotation services like Amazon’s Mechanical Turk (AMT) are cheap and fast
ways to evaluate systems and provide categorical annotations for training data. Unfortunately, …

Much recent work seeks to evaluate values and opinions in large language models (LLMs)
using multiple-choice surveys and questionnaires. Most of this work is motivated by concerns …

Recently, there has been an increased interest in demographically grounded bias in natural
language processing (NLP) applications. Much of the recent work has focused on …

Topic models extract groups of words from documents, whose interpretation as a topic
hopefully allows for a better understanding of the data. However, the resulting word groups are …

An increasing number of natural language processing papers address the effect of bias on
predictions, introducing mitigation techniques at different parts of the standard NLP pipeline (…

The open-ended nature of language generation makes the evaluation of autoregressive
large language models (LLMs) challenging. One common evaluation approach uses multiple-…

Many tasks in Natural Language Processing (NLP) and Computer Vision (CV) offer
evidence that humans disagree, from objective tasks such as part-of-speech tagging to more …