Umeå universitets logga

umu.sePublikationer
Ändra sökning
Länk till posten
Permanent länk

Direktlänk
Devinney, Hannah
Publikationer (8 of 8) Visa alla publikationer
Devinney, H. (2024). Gender and representation: investigations of bias in natural language processing. (Doctoral dissertation). Umeå: Umeå University
Öppna denna publikation i ny flik eller fönster >>Gender and representation: investigations of bias in natural language processing
2024 (Engelska)Doktorsavhandling, monografi (Övrigt vetenskapligt)
Alternativ titel[sv]
Genus och representation : studier av social bias i språkteknologi
Abstract [en]

Natural Language Processing (NLP) technologies are a part of our every day realities. They come in forms we can easily see as ‘language technologies’ (auto-correct, translation services, search results) as well as those that fly under our radar (social media algorithms, 'suggested reading' recommendations on news sites, spam filters). NLP fuels many other tools under the Artificial Intelligence umbrella – such as algorithms approving for loan applications – which can have major material effects on our lives. As large language models like ChatGPT have become popularized, we are also increasingly exposed to machine-generated texts.

Machine Learning (ML) methods, which most modern NLP tools rely on, replicate patterns in their training data. Typically, these language data are generated by humans, and contain both overt and underlying patterns that we consider socially undesirable, comprising stereotypes and other reflections of human prejudice. Such patterns (often termed 'bias') are picked up and repeated, or even made more extreme, by ML systems. Thus, NLP technologies become a part of the linguistic landscapes in which we humans transmit stereotypes and act on our prejudices. They may participate in this transmission by, for example, translating nurses as women (and doctors as men) or systematically preferring to suggest promoting men over women. These technologies are tools in the construction of power asymmetries not only through the reinforcement of hegemony, but also through the distribution of material resources when they are included in decision-making processes such as screening job applications.

This thesis explores gendered biases, trans and nonbinary inclusion, and queer representation within NLP through a feminist and intersectional lens. Three key areas are investigated: the ways in which “gender” is theorized and operationalized by researchers investigating gender bias in NLP; gendered associations within datasets used for training language technologies; and the representation of queer (particularly trans and nonbinary) identities in the output of both low-level NLP models and large language models (LLMs). 

The findings indicate that nonbinary people/genders are erased by both bias in NLP tools/datasets, and by research/ers attempting to address gender biases. Men and women are also held to cisheteronormative standards (and stereotypes), which is particularly problematic when considering the intersection of gender and sexuality. Although it is possible to mitigate some of these issues in particular circumstances, such as addressing erasure by adding more examples of nonbinary language to training data, the complex nature of the socio-technical landscape which NLP technologies are a part of means that simple fixes may not always be sufficient. Additionally, it is important that ways of measuring and mitigating 'bias' remain flexible, as our understandings of social categories, stereotypes and other undesirable norms, and 'bias' itself will shift across contexts such as time and linguistic setting. 

Abstract [sv]

Nuförtiden möter vi dagligen språkteknologi i olika former. Ibland är det tydligt för oss att detta sker, till exempel när vi använder maskinöversättning. Andra gånger är det svårare att upptäcka, som när sociala medier rekommenderar oss inlägg. Språkteknologi ligger också till grund för större AI-system, som till exempel kan användas för att bevilja eller avslå låneansökningar och därmed ha stora materiella effekter på våra liv. I takt med att ChatGPT och andra stora språkmodeller blir mer populära kommer vi också att konfronteras med fler och fler maskingenererade texter.       

Maskininlärningsmetoder, som de flesta av dessa verktyg förlitar sig på idag, upprepar mönster de 'ser' i sin träningsdata. Vanligtvis är detta språkdata som människor har skrivit eller talat, så förutom saker som meningsstruktur innehåller den också information om hur vi konstruerar vårt samhälle. Detta inkluderar även stereotyper och andra fördomar. Vi kallar dessa mönster för 'social bias' och de upprepas, eller till och med förvärras, av maskininlärningssystem. När språkteknologi blir en del av vårt språkliga sammanhang blir de också delaktiga i att föra vidare stereotyper genom att till exempel anta att sjuksköterskor är kvinnor och läkare män, eller systematiskt föreslå män framför kvinnor för befordran. Tekniken blir därmed ett verktyg som samhället använder för att bygga upp makt -- och maktskillnader -- genom att sprida och normalisera orättvisa idéer samt genom att bidra till orättvisa resursfördelningar.       

Den här avhandlingen utforskar sociala fördomar om kön och genus, inkludering av trans- och ickebinära personer samt queer representation i språkteknologier genom en feministisk och intersektionell lins. Tre frågor ställs: Hur tänker forskare på och mäter 'genus' när de undersöker 'genusbias' i språkteknologi? Vilka könsstereotyper finns i data som används för att träna språkteknologiska modeller? Hur representeras queera (särskilt trans- och ickebinära) människor, kroppar och erfarenheter i produktionen av dessa teknologier? Avhandlingen finner att ickebinära personer osynliggörs av fördomar i såväl modeller som data, men också av forskare som vill ta itu med könsfördomar. Män och kvinnor reduceras till cisheteronormativa roller och stereotyper, med litet utrymme att vara en individ bortom kön. Vi kan mildra några av dessa problem, till exempel genom att lägga till mer ickebinärt språk i träningsdatan, men fullständiga lösningar är svåra att uppnå på grund av det komplexa samspelet mellan samhälle och teknik. Dessutom måste vi förbli flexibla, eftersom vår förståelse av samhället, stereotyper och 'bias' i sig skiftar över tid och med sammanhanget.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå University, 2024. s. 173
Serie
Report / UMINF, ISSN 0348-0542
Nyckelord
NLP, natural language processing, gender bias, social impact of AI, gendered pronouns, neopronouns, gender studies, topic modeling
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling) Genusstudier
Forskningsämne
datalogi; datorlingvistik; genusvetenskap
Identifikatorer
urn:nbn:se:umu:diva-222468 (URN)978-91-8070-337-6 (ISBN)978-91-8070-336-9 (ISBN)
Disputation
2024-04-18, MIT.A.121, MIT-huset, Umeå, 13:00 (Engelska)
Opponent
Handledare
Tillgänglig från: 2024-03-27 Skapad: 2024-03-19 Senast uppdaterad: 2024-03-21Bibliografiskt granskad
Aler Tubella, A., Coelho Mollo, D., Dahlgren, A., Devinney, H., Dignum, V., Ericson, P., . . . Nieves, J. C. (2023). ACROCPoLis: a descriptive framework for making sense of fairness. In: FAccT '23: Proceedings of the 2023 ACM conference on fairness, accountability, and transparency. Paper presented at 2023 ACM Conference on Fairness, Accountability, and Transparency, Chicago, Illinois, USA, June 12-15, 2023 (pp. 1014-1025). ACM Digital Library
Öppna denna publikation i ny flik eller fönster >>ACROCPoLis: a descriptive framework for making sense of fairness
Visa övriga...
2023 (Engelska)Ingår i: FAccT '23: Proceedings of the 2023 ACM conference on fairness, accountability, and transparency, ACM Digital Library, 2023, s. 1014-1025Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Fairness is central to the ethical and responsible development and use of AI systems, with a large number of frameworks and formal notions of algorithmic fairness being available. However, many of the fairness solutions proposed revolve around technical considerations and not the needs of and consequences for the most impacted communities. We therefore want to take the focus away from definitions and allow for the inclusion of societal and relational aspects to represent how the effects of AI systems impact and are experienced by individuals and social groups. In this paper, we do this by means of proposing the ACROCPoLis framework to represent allocation processes with a modeling emphasis on fairness aspects. The framework provides a shared vocabulary in which the factors relevant to fairness assessments for different situations and procedures are made explicit, as well as their interrelationships. This enables us to compare analogous situations, to highlight the differences in dissimilar situations, and to capture differing interpretations of the same situation by different stakeholders.

Ort, förlag, år, upplaga, sidor
ACM Digital Library, 2023
Nyckelord
Algorithmic fairness; socio-technical processes; social impact of AI; responsible AI
Nationell ämneskategori
Systemvetenskap, informationssystem och informatik
Forskningsämne
datalogi
Identifikatorer
urn:nbn:se:umu:diva-209705 (URN)10.1145/3593013.3594059 (DOI)2-s2.0-85163594710 (Scopus ID)978-1-4503-7252-7 (ISBN)
Konferens
2023 ACM Conference on Fairness, Accountability, and Transparency, Chicago, Illinois, USA, June 12-15, 2023
Tillgänglig från: 2023-06-13 Skapad: 2023-06-13 Senast uppdaterad: 2023-07-18Bibliografiskt granskad
Björklund, H. & Devinney, H. (2023). Computer, enhence: POS-tagging improvements for nonbinary pronoun use in Swedish. In: Proceedings of the third workshop on language technology for equality, diversity, inclusion: . Paper presented at Third Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI-2023) at RANLP 2023, Varna, Bulgaria, September 7, 2023 (pp. 54-61). The Association for Computational Linguistics
Öppna denna publikation i ny flik eller fönster >>Computer, enhence: POS-tagging improvements for nonbinary pronoun use in Swedish
2023 (Engelska)Ingår i: Proceedings of the third workshop on language technology for equality, diversity, inclusion, The Association for Computational Linguistics , 2023, s. 54-61Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Part of Speech (POS) taggers for Swedish routinely fail for the third person gender-neutral pronoun hen, despite the fact that it has been a well-established part of the Swedish language since at least 2014. In addition to simply being a form of gender bias, this failure can have negative effects on other tasks relying on POS information. We demonstrate the usefulness of semi-synthetic augmented datasets in a case study, retraining a POS tagger to correctly recognize hen as a personal pronoun. We evaluate our retrained models for both tag accuracy and on a downstream task (dependency parsing) in a classicial NLP pipeline.

Our results show that adding such data works to correct for the disparity in performance. The accuracy rate for identifying hen as a pronoun can be brought up to acceptable levels with only minor adjustments to the tagger’s vocabulary files. Performance parity to gendered pronouns can be reached after retraining with only a few hundred examples. This increase in POS tag accuracy also results in improvements for dependency parsing sentences containing hen.

Ort, förlag, år, upplaga, sidor
The Association for Computational Linguistics, 2023
Nyckelord
Part-of-Speech, gendered pronouns, neopronouns
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling)
Forskningsämne
datorlingvistik
Identifikatorer
urn:nbn:se:umu:diva-213782 (URN)10.26615/978-954-452-084-7_008 (DOI)2-s2.0-85184990283 (Scopus ID)978-954-452-084-7 (ISBN)
Konferens
Third Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI-2023) at RANLP 2023, Varna, Bulgaria, September 7, 2023
Forskningsfinansiär
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Tillgänglig från: 2023-09-26 Skapad: 2023-09-26 Senast uppdaterad: 2024-02-27Bibliografiskt granskad
Devinney, H., Eklund, A., Ryazanov, I. & Cai, J. (2023). Developing a multilingual corpus of wikipedia biographies. In: Ruslan Mitkov; Maria Kunilovskaya; Galia Angelova (Ed.), International conference. Recent advances in natural language processing 2023, large language models for natural language processing: proceedings. Paper presented at 14th international conference on Recent Advances in Natural Language Processing 2023, Varna, Bulgaria, September 4-6, 2023.. Shoumen, Bulgaria: Incoma ltd., Article ID 2023.ranlp-1.32.
Öppna denna publikation i ny flik eller fönster >>Developing a multilingual corpus of wikipedia biographies
2023 (Engelska)Ingår i: International conference. Recent advances in natural language processing 2023, large language models for natural language processing: proceedings / [ed] Ruslan Mitkov; Maria Kunilovskaya; Galia Angelova, Shoumen, Bulgaria: Incoma ltd. , 2023, artikel-id 2023.ranlp-1.32Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

For many languages, Wikipedia is the mostaccessible source of biographical information. Studying how Wikipedia describes the lives ofpeople can provide insights into societal biases, as well as cultural differences more generally. We present a method for extracting datasetsof Wikipedia biographies. The accompanying codebase is adapted to English, Swedish, Russian, Chinese, and Farsi, and is extendable to other languages. We present an exploratory analysis of biographical topics and gendered patterns in four languages using topic modelling and embedding clustering. We find similarities across languages in the types of categories present, with the distribution of biographies concentrated in the language’s core regions. Masculine terms are over-represented and spread out over a wide variety of topics. Feminine terms are less frequent and linked to more constrained topics. Non-binary terms are nearly non-represented.

Ort, förlag, år, upplaga, sidor
Shoumen, Bulgaria: Incoma ltd., 2023
Serie
International conference Recent advances in natural language processing, ISSN 2603-2813 ; 2023
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling)
Forskningsämne
datorlingvistik
Identifikatorer
urn:nbn:se:umu:diva-213781 (URN)10.26615/978-954-452-092-2_032 (DOI)2-s2.0-85179178058 (Scopus ID)978-954-452-092-2 (ISBN)
Konferens
14th international conference on Recent Advances in Natural Language Processing 2023, Varna, Bulgaria, September 4-6, 2023.
Tillgänglig från: 2023-11-10 Skapad: 2023-11-10 Senast uppdaterad: 2023-12-22Bibliografiskt granskad
Björklund, H. & Devinney, H. (2022). Improving Swedish part-of-speech tagging for hen. In: : . Paper presented at Swedish Language Technology Conference 2022, Stockholm, Sweden, November 23-25, 2022.
Öppna denna publikation i ny flik eller fönster >>Improving Swedish part-of-speech tagging for hen
2022 (Engelska)Konferensbidrag, Enbart muntlig presentation (Refereegranskat)
Abstract [en]

Despite the fact that the gender-neutral pro-noun hen was officially added to the Swedish language in 2014, state of the art part of speech taggers still routinely fail to identify it as a pronoun. We retrain both efselab and spaCy models with augmented (semi-synthetic) data, where instances of gendered pronouns are replaced by hen to correct for the lack of representation in the original training data. Our results show that adding such data works to correct for the disparity in performance

Nyckelord
Part-of-Speech, gendered pronouns, neopronouns
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling)
Forskningsämne
datorlingvistik
Identifikatorer
urn:nbn:se:umu:diva-201268 (URN)
Konferens
Swedish Language Technology Conference 2022, Stockholm, Sweden, November 23-25, 2022
Tillgänglig från: 2022-11-24 Skapad: 2022-11-24 Senast uppdaterad: 2022-11-28Bibliografiskt granskad
Devinney, H., Björklund, J. & Björklund, H. (2022). Theories of Gender in Natural Language Processing. In: Proceedings of the fifth annual ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT'22): . Paper presented at ACM FAccT Conference 2022, Conference on Fairness, Accountability, and Transparency, Hybrid via Seoul, Soth Korea, June 21-14, 2022.
Öppna denna publikation i ny flik eller fönster >>Theories of Gender in Natural Language Processing
2022 (Engelska)Ingår i: Proceedings of the fifth annual ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT'22), 2022Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The rise of concern around Natural Language Processing (NLP) technologies containing and perpetuating social biases has led to a rich and rapidly growing area of research. Gender bias is one of the central biases being analyzed, but to date there is no comprehensive analysis of how “gender” is theorized in the field. We survey nearly 200 articles concerning gender bias in NLP to discover how the field conceptualizes gender both explicitly (e.g. through definitions of terms) and implicitly (e.g. through how gender is operationalized in practice). In order to get a better idea of emerging trajectories of thought, we split these articles into two sections by time.

We find that the majority of the articles do not make their theo- rization of gender explicit, even if they clearly define “bias.” Almost none use a model of gender that is intersectional or inclusive of non- binary genders; and many conflate sex characteristics, social gender, and linguistic gender in ways that disregard the existence and expe- rience of trans, nonbinary, and intersex people. There is an increase between the two time-sections in statements acknowledging that gender is a complicated reality, however, very few articles manage to put this acknowledgment into practice. In addition to analyzing these findings, we provide specific recommendations to facilitate interdisciplinary work, and to incorporate theory and methodol- ogy from Gender Studies. Our hope is that this will produce more inclusive gender bias research in NLP.

Nyckelord
natural language processing, gender bias, gender studies
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling) Genusstudier
Forskningsämne
datalogi; genusvetenskap
Identifikatorer
urn:nbn:se:umu:diva-194742 (URN)10.1145/3531146.3534627 (DOI)2-s2.0-85133018925 (Scopus ID)
Konferens
ACM FAccT Conference 2022, Conference on Fairness, Accountability, and Transparency, Hybrid via Seoul, Soth Korea, June 21-14, 2022
Anmärkning

Alternative title: "Theories of 'Gender' in NLP Bias Research"

Tillgänglig från: 2022-05-16 Skapad: 2022-05-16 Senast uppdaterad: 2023-03-24
Devinney, H., Björklund, J. & Björklund, H. (2020). Crime and Relationship: Exploring Gender Bias in NLP Corpora. In: : . Paper presented at SLTC 2020 – The Eighth Swedish Language Technology Conference, 25–27 November 2020, Online.
Öppna denna publikation i ny flik eller fönster >>Crime and Relationship: Exploring Gender Bias in NLP Corpora
2020 (Engelska)Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Gender bias in natural language processing (NLP) tools, deriving from implicit human bias embedded in language data, is an important and complicated problem on the road to fair algorithms. We leverage topic modeling to retrieve documents associated with particular gendered categories, and discuss how exploring these documents can inform our understanding of the corpora we may use to train NLP tools. This is a starting point for challenging the systemic power structures and producing a justice-focused approach to NLP.

Nyckelord
gender bias, topic modeling
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling) Genusstudier
Forskningsämne
datalogi; genusvetenskap
Identifikatorer
urn:nbn:se:umu:diva-177583 (URN)
Konferens
SLTC 2020 – The Eighth Swedish Language Technology Conference, 25–27 November 2020, Online
Projekt
EQUITBL
Tillgänglig från: 2020-12-14 Skapad: 2020-12-14 Senast uppdaterad: 2021-01-14Bibliografiskt granskad
Devinney, H., Björklund, J. & Björklund, H. (2020). Semi-Supervised Topic Modeling for Gender Bias Discovery in English and Swedish. In: Marta R. Costa-jussà, Christian Hardmeier, Will Radford, Kellie Webster (Ed.), Proceedings of the Second Workshop on Gender Bias in Natural Language Processing: . Paper presented at GeBNLP2020, COLING'2020 – The 28th International Conference on Computational Linguistics, December 8-13, 2020, Online (pp. 79-92). Association for Computational Linguistics
Öppna denna publikation i ny flik eller fönster >>Semi-Supervised Topic Modeling for Gender Bias Discovery in English and Swedish
2020 (Engelska)Ingår i: Proceedings of the Second Workshop on Gender Bias in Natural Language Processing / [ed] Marta R. Costa-jussà, Christian Hardmeier, Will Radford, Kellie Webster, Association for Computational Linguistics, 2020, s. 79-92Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Gender bias has been identified in many models for Natural Language Processing, stemming from implicit biases in the text corpora used to train the models. Such corpora are too large to closely analyze for biased or stereotypical content. Thus, we argue for a combination of quantitative and qualitative methods, where the quantitative part produces a view of the data of a size suitable for qualitative analysis. We investigate the usefulness of semi-supervised topic modeling for the detection and analysis of gender bias in three corpora (mainstream news articles in English and Swedish, and LGBTQ+ web content in English). We compare differences in topic models for three gender categories (masculine, feminine, and nonbinary or neutral) in each corpus. We find that in all corpora, genders are treated differently and that these differences tend to correspond to hegemonic ideas of gender.

Ort, förlag, år, upplaga, sidor
Association for Computational Linguistics, 2020
Nyckelord
gender bias, topic modelling
Nationell ämneskategori
Språkteknologi (språkvetenskaplig databehandling) Genusstudier
Forskningsämne
datalogi; genusvetenskap
Identifikatorer
urn:nbn:se:umu:diva-177576 (URN)
Konferens
GeBNLP2020, COLING'2020 – The 28th International Conference on Computational Linguistics, December 8-13, 2020, Online
Projekt
EQUITBL
Tillgänglig från: 2020-12-14 Skapad: 2020-12-14 Senast uppdaterad: 2021-01-14Bibliografiskt granskad
Organisationer

Sök vidare i DiVA

Visa alla publikationer