Fairness is central to the ethical and responsible development and use of AI systems, with a large number of frameworks and formal notions of algorithmic fairness being available. However, many of the fairness solutions proposed revolve around technical considerations and not the needs of and consequences for the most impacted communities. We therefore want to take the focus away from definitions and allow for the inclusion of societal and relational aspects to represent how the effects of AI systems impact and are experienced by individuals and social groups. In this paper, we do this by means of proposing the ACROCPoLis framework to represent allocation processes with a modeling emphasis on fairness aspects. The framework provides a shared vocabulary in which the factors relevant to fairness assessments for different situations and procedures are made explicit, as well as their interrelationships. This enables us to compare analogous situations, to highlight the differences in dissimilar situations, and to capture differing interpretations of the same situation by different stakeholders.
Part of Speech (POS) taggers for Swedish routinely fail for the third person gender-neutral pronoun hen, despite the fact that it has been a well-established part of the Swedish language since at least 2014. In addition to simply being a form of gender bias, this failure can have negative effects on other tasks relying on POS information. We demonstrate the usefulness of semi-synthetic augmented datasets in a case study, retraining a POS tagger to correctly recognize hen as a personal pronoun. We evaluate our retrained models for both tag accuracy and on a downstream task (dependency parsing) in a classicial NLP pipeline.
Our results show that adding such data works to correct for the disparity in performance. The accuracy rate for identifying hen as a pronoun can be brought up to acceptable levels with only minor adjustments to the tagger’s vocabulary files. Performance parity to gendered pronouns can be reached after retraining with only a few hundred examples. This increase in POS tag accuracy also results in improvements for dependency parsing sentences containing hen.
Despite the fact that the gender-neutral pro-noun hen was officially added to the Swedish language in 2014, state of the art part of speech taggers still routinely fail to identify it as a pronoun. We retrain both efselab and spaCy models with augmented (semi-synthetic) data, where instances of gendered pronouns are replaced by hen to correct for the lack of representation in the original training data. Our results show that adding such data works to correct for the disparity in performance
Nuförtiden möter vi dagligen språkteknologi i olika former. Ibland är det tydligt för oss att detta sker, till exempel när vi använder maskinöversättning. Andra gånger är det svårare att upptäcka, som när sociala medier rekommenderar oss inlägg. Språkteknologi ligger också till grund för större AI-system, som till exempel kan användas för att bevilja eller avslå låneansökningar och därmed ha stora materiella effekter på våra liv. I takt med att ChatGPT och andra stora språkmodeller blir mer populära kommer vi också att konfronteras med fler och fler maskingenererade texter.
Maskininlärningsmetoder, som de flesta av dessa verktyg förlitar sig på idag, upprepar mönster de 'ser' i sin träningsdata. Vanligtvis är detta språkdata som människor har skrivit eller talat, så förutom saker som meningsstruktur innehåller den också information om hur vi konstruerar vårt samhälle. Detta inkluderar även stereotyper och andra fördomar. Vi kallar dessa mönster för 'social bias' och de upprepas, eller till och med förvärras, av maskininlärningssystem. När språkteknologi blir en del av vårt språkliga sammanhang blir de också delaktiga i att föra vidare stereotyper genom att till exempel anta att sjuksköterskor är kvinnor och läkare män, eller systematiskt föreslå män framför kvinnor för befordran. Tekniken blir därmed ett verktyg som samhället använder för att bygga upp makt -- och maktskillnader -- genom att sprida och normalisera orättvisa idéer samt genom att bidra till orättvisa resursfördelningar.
Den här avhandlingen utforskar sociala fördomar om kön och genus, inkludering av trans- och ickebinära personer samt queer representation i språkteknologier genom en feministisk och intersektionell lins. Tre frågor ställs: Hur tänker forskare på och mäter 'genus' när de undersöker 'genusbias' i språkteknologi? Vilka könsstereotyper finns i data som används för att träna språkteknologiska modeller? Hur representeras queera (särskilt trans- och ickebinära) människor, kroppar och erfarenheter i produktionen av dessa teknologier? Avhandlingen finner att ickebinära personer osynliggörs av fördomar i såväl modeller som data, men också av forskare som vill ta itu med könsfördomar. Män och kvinnor reduceras till cisheteronormativa roller och stereotyper, med litet utrymme att vara en individ bortom kön. Vi kan mildra några av dessa problem, till exempel genom att lägga till mer ickebinärt språk i träningsdatan, men fullständiga lösningar är svåra att uppnå på grund av det komplexa samspelet mellan samhälle och teknik. Dessutom måste vi förbli flexibla, eftersom vår förståelse av samhället, stereotyper och 'bias' i sig skiftar över tid och med sammanhanget.
Gender bias in natural language processing (NLP) tools, deriving from implicit human bias embedded in language data, is an important and complicated problem on the road to fair algorithms. We leverage topic modeling to retrieve documents associated with particular gendered categories, and discuss how exploring these documents can inform our understanding of the corpora we may use to train NLP tools. This is a starting point for challenging the systemic power structures and producing a justice-focused approach to NLP.
Gender bias has been identified in many models for Natural Language Processing, stemming from implicit biases in the text corpora used to train the models. Such corpora are too large to closely analyze for biased or stereotypical content. Thus, we argue for a combination of quantitative and qualitative methods, where the quantitative part produces a view of the data of a size suitable for qualitative analysis. We investigate the usefulness of semi-supervised topic modeling for the detection and analysis of gender bias in three corpora (mainstream news articles in English and Swedish, and LGBTQ+ web content in English). We compare differences in topic models for three gender categories (masculine, feminine, and nonbinary or neutral) in each corpus. We find that in all corpora, genders are treated differently and that these differences tend to correspond to hegemonic ideas of gender.
The rise of concern around Natural Language Processing (NLP) technologies containing and perpetuating social biases has led to a rich and rapidly growing area of research. Gender bias is one of the central biases being analyzed, but to date there is no comprehensive analysis of how “gender” is theorized in the field. We survey nearly 200 articles concerning gender bias in NLP to discover how the field conceptualizes gender both explicitly (e.g. through definitions of terms) and implicitly (e.g. through how gender is operationalized in practice). In order to get a better idea of emerging trajectories of thought, we split these articles into two sections by time.
We find that the majority of the articles do not make their theo- rization of gender explicit, even if they clearly define “bias.” Almost none use a model of gender that is intersectional or inclusive of non- binary genders; and many conflate sex characteristics, social gender, and linguistic gender in ways that disregard the existence and expe- rience of trans, nonbinary, and intersex people. There is an increase between the two time-sections in statements acknowledging that gender is a complicated reality, however, very few articles manage to put this acknowledgment into practice. In addition to analyzing these findings, we provide specific recommendations to facilitate interdisciplinary work, and to incorporate theory and methodol- ogy from Gender Studies. Our hope is that this will produce more inclusive gender bias research in NLP.
Despite concerns that Large Language Models (LLMs) are vectors for reproducing and ampli- fying social biases such as sexism, transpho- bia, islamophobia, and racism, there is a lack of work qualitatively analyzing how such pat- terns of bias are generated by LLMs. We use mixed-methods approaches and apply a femi- nist, intersectional lens to the problem across two language domains, Swedish and English, by generating narrative texts using LLMs. We find that hegemonic norms are consistently re- produced; dominant identities are often treated as ‘default’; and discussion of identity itself may be considered ‘inappropriate’ by the safety features applied to some LLMs. Due to the dif- fering behaviors of models, depending both on their design and the language they are trained on, we observe that strategies of identifying “bias” must be adapted to individual models and their socio-cultural contexts.
For many languages, Wikipedia is the mostaccessible source of biographical information. Studying how Wikipedia describes the lives ofpeople can provide insights into societal biases, as well as cultural differences more generally. We present a method for extracting datasetsof Wikipedia biographies. The accompanying codebase is adapted to English, Swedish, Russian, Chinese, and Farsi, and is extendable to other languages. We present an exploratory analysis of biographical topics and gendered patterns in four languages using topic modelling and embedding clustering. We find similarities across languages in the types of categories present, with the distribution of biographies concentrated in the language’s core regions. Masculine terms are over-represented and spread out over a wide variety of topics. Feminine terms are less frequent and linked to more constrained topics. Non-binary terms are nearly non-represented.