Lai, Catherine

Hall-Lew, Lauren

Markl, Nina

Economic and Social Research Council (ESRC)

2023-12-12T12:16:33Z

2023-12-12T12:16:33Z

2023-12-12

In this thesis, I situate the impacts of automatic speech recognition systems in relation to sociolinguistic theory (in particular drawing on concepts of language variation, language ideology and language policy) and contemporary debates in AI ethics (especially regarding algorithmic bias and fairness). In recent years, automatic speech recognition systems, alongside other language technologies, have been adopted by a growing number of users and have been embedded in an increasing number of algorithmic systems. This expansion into new application domains and language varieties can be understood as an expansion into new sociolinguistic contexts. In this thesis, I am interested in how automatic speech recognition tools interact with this sociolinguistic context, and how they affect speakers, speech communities and their language varieties. Focussing on commercial automatic speech recognition systems for British Englishes, I first explore the extent and consequences of performance differences of these systems for different user groups depending on their linguistic background. When situating this predictive bias within the wider sociolinguistic context, it becomes apparent that these systems reproduce and potentially entrench existing linguistic discrimination and could therefore cause direct and indirect harms to already marginalised speaker groups. To understand the benefits and potentials of automatic transcription tools, I highlight two case studies: transcribing sociolinguistic data in English and transcribing personal voice messages in isiXhosa. The central role of the sociolinguistic context in developing these tools is emphasised in this comparison. Design choices, such as the choice of training data, are particularly consequential because they interact with existing processes of language standardisation. To understand the impacts of these choices, and the role of the developers making them better, I draw on theory from language policy research and critical data studies. These conceptual frameworks are intended to help practitioners and researchers in anticipating and mitigating predictive bias and other potential harms of speech technologies. Beyond looking at individual choices, I also investigate the discourses about language variation and linguistic diversity deployed in the context of language technologies. These discourses put forward by researchers, developers and commercial providers not only have a direct effect on the wider sociolinguistic context, but they also highlight how this context (e.g., existing beliefs about language(s)) affects technology development. Finally, I explore ways of building better automatic speech recognition tools, focussing in particular on well-documented, naturalistic and diverse benchmark datasets. However, inclusive datasets are not necessarily a panacea, as they still raise important questions about the nature of linguistic data and language variation (especially in relation to identity), and may not mitigate or prevent all potential harms of automatic speech recognition systems as embedded in larger algorithmic systems and sociolinguistic contexts.

https://hdl.handle.net/1842/41277

http://dx.doi.org/10.7488/era/4013

en

The University of Edinburgh

Nina Markl (2022b). “Language Variation and Algorithmic Bias: Understanding Algorithmic Bias in British English Automatic Speech Recognition”. In: 2022 ACM Conference on Fairness, Accountability, and Transparency. FAccT ’22. Seoul, Republic of Korea: Association for Computing Machinery, pp. 521–534. DOI: 10.1145/3531146.3533117

Nina Markl (2022a). “(Commercial) Automatic Speech Recognition as a Tool in Sociolinguistic Research”. In: University of Pennsylvania Working Papers in Linguistics 28.2. URL: https://repository.upenn.edu/pwpl/vol28/iss2/11

Nina Markl, Electra Wallington, Ondrej Klejch, Thomas Reitmaier, Gavin Bailey, Jennifer Pearson, Matt Jones, Simon Robinson, and Peter Bell (2023). “Automatic transcription and (de)standardisation”. English. In: Proceedings - SIGUL 2023, 2nd Annual Meeting of the Special Interest Group on Under-resourced Languages. URL: https://sigul-2023.ilc.cnr.it/wp-content/uploads/2023/08/9_Paper.pdf

Nina Markl (2022c). “Mind the data gap(s): Investigating power in speech and language datasets”. In: Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion. Dublin, Ireland: Association for Computational Linguistics, pp. 1–12. URL: https://aclanthology.org/2022.ltedi-1.1

Nina Markl and Stephen Joseph McNulty (2022). “Language technology practitioners as language managers: arbitrating data bias and predictive bias in ASR”. in: Proceedings of the Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association, pp. 6328–6339. URL: https : / / aclanthology . org/2022.lrec-1.680

Nina Markl and Catherine Lai (2023). “Everyone has an accent”. In: Proc. INTERSPEECH 2023, pp. 4424–4427. DOI: 10.21437/Interspeech.2023-1847

Nina Markl and Catherine Lai (2021). “Context-sensitive evaluation of automatic speech recognition: considering user experience & language variation”. In: Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing. Online: Association for Computational Linguistics, pp. 34–40. URL: https: //aclanthology.org/2021.hcinlp-1.6

Ramon Sanabria, Nikolay Bogoychev, Nina Markl, Andrea Carmantini, Klejch Ondřej, and Peter Bell (2023). “The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR”. in: ICASSP 2023, pp. 1–5. DOI: 10.1109/ ICASSP49357.2023.10095057

Hall-Lew, Lauren, Claire Cowie, Catherine Lai, Nina Markl, Stephen Joseph McNulty, Shan-Jan Sarah Liu, Clare Llewellyn, Beatrice Alex, Zuzana Elliott, and Anita Klingler (2022). “The Lothian Diary Project: sociolinguistic methods during the COVID-19 lockdown”. In: Linguistics Vanguard 8.s3, pp. 321–330. DOI: doi:10.1515/lingvan-2021-0053

Reitmaier, Thomas, Electra Wallington, Ondřej Klejch, Nina Markl, Lea-Marie Lam-Yee-Mui, Jennifer Pearson, Matt Jones, Peter Bell, and Simon Robinson (2023). “Situating Automatic Speech Recognition Development within Communities of Under-heard Language Speakers”. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23. New York, NY, USA: Association for Computing Machinery. DOI: 10.1145/ 3544548.3581385.

automatic speech recognition

linguistic background

voice technology development

standard language

marginalised groups

representative speech datasets

inclusive design

Language variation, automatic speech recognition and algorithmic bias

Thesis or Dissertation

Doctoral

PhD Doctor of Philosophy