MISGENDERED: Limits of Large Language Models in Understanding Pronouns

Tamanna Hossain
Sameer Singh
ACL (2023)

Abstract

Gender bias in language technologies has been widely studied, but research has mostly been restricted to a binary paradigm of gender.
It is important to also consider non-binary gender identities, as excluding them can cause further harm to an already marginalized group.
One way in which English-speaking individuals linguistically encode their gender identity is through third-person personal pronoun declarations.
This is often done using two or more pronoun forms, e.g., \textit{xe/xem}, or \textit{xe/xem/xyr}.
In this paper, we comprehensively evaluate state-of-the-art language models for their ability to correctly use declared third-person personal pronouns.
As far as we are aware, we are the first to do so.
We evaluate language models in both zero-shot and few-shot settings.
Models are still far from zero-shot gendering non-binary individuals accurately, and most also struggle with correctly using gender-neutral pronouns (singular \textit{they, them, their} etc.).
This poor performance may be due to the lack of representation of non-binary pronouns in pre-training corpora, and some memorized associations between pronouns and names.
We find an overall improvement in performance for non-binary pronouns when using in-context learning, demonstrating that language models with few-shot capabilities can adapt to using declared pronouns correctly.