Oh, the humanity! A human-centric approach to social bias research in natural language processing

Abstract

Much current research into social bias in Natural Language Processing (NLP) -- that is, the tendency for NLP technologies to reflect human biases such as sexism and homophobia in the relative probability of different outputs -- suffers from relying on a superficial understanding of the problem. The issue of social bias is treated as a mathematical kink that needs to be ironed out, after which the harm that the model does will be irrefutably reduced -- a form of algorithmic idealism. Social bias is seen as an unfortunate result of "dirty data'' and "data imbalance'', and practitioners typically focus on addressing social bias through changes at training or inference to counteract these data issues. Little regard is given to the human aspect: to the myriad normative choices made by those who develop these systems; the beliefs of those who deploy them; nor to the response of those impacted by these technologies, all of which will influence how social bias is actually experienced. Operationalising bias as a quantifiable metric allows for at-scale evaluation that keeps pace with the rapid development of new NLP technologies. However, I argue this superficial understanding of social bias will lead us to superficial and ultimately ineffective solutions, which ignore the role of human behaviour in determining the harm done by technology. As I demonstrate, heuristic attempts at social bias mitigation often end up doing more harm than good.

In my thesis, I advocate for a human-centric approach to measuring and mitigating social bias in NLP, one which focuses on human choices, human identities and human behaviour, to give a more complete understanding of the true impact of NLP technologies. A human-centric approach treats social bias as a socio-technical problem, and casts its net widely over a broad range of stakeholders, sources of bias, and demographic attributes. My proposed approach is underpinned by five maxims: see technology as part of a socio-technical system; consider many sources of bias; focus on the impact on people and how they respond; be driven by social science theory and community knowledge; address a broad range of demographics.

I present my work as four case studies across three tasks which demonstrate the benefits of this approach. I consider harms against marginalised (primarily queer) identities through social bias in sentiment analysis tools, text-to-image (TTI) models and social media recommender and moderation algorithms (namely on TikTok), finding that heuristic attempts to reduce social bias often do more harm than good, and that the public form complex beliefs about NLP technologies. In all my work, I address social biases in publicly available or public facing tools, as these typically have a broader impact than state of the art models. I focus primarily on harms done to the LGBTQ+ community, in part because of personal relevance, but also because it provides an opportunity to demonstrate the benefits of an approach that considers demographic qualities beyond a binary. There are no binaries in nature -- in human identity -- yet much social bias research hinges on treating demographics as such. In doing so I contribute significantly to our understanding of queerphobia in NLP.

Ultimately I argue -- despite the title of this thesis -- that the best approach to social bias research is one that switches focus from social bias to real-world harms. Social bias has been used as a proxy for harm, but as I argue, it is often a very poor one. Changing our focus to harms necessitates defining specific use contexts (real or imagined). The "meaning'' of a difference in probability will be context dependent, as will how this difference is interpreted by those impacted by the model. I am far from the first to critique current practices, and the disconnect between social bias and harm; I amplify the message, and enrich it with five clear maxims that improve the validity of social bias research. To leave the human context out of the equation when measuring harm is nonsensical, yet for too long the field of NLP has attempted to do exactly that. Oh, the humanity!