Arya D. McCarthy
Arya McCarthy is a Research Scientist at Google DeepMind, where he expands the multilingual capabilities of Gemini for a global audience. His research focuses on bridging the gap between languages through structure-grounded machine translation, speech algorithms, and model adaptation for specialized domains.
Learn more: https://aryamccarthy.github.io/
Learn more: https://aryamccarthy.github.io/
Research Areas
Authored Publications
Sort By
LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domain
Joel Niklaus
Lucia Zheng
Christopher Hahn
Brian Rosen
Peter Henderson
Daniel E. Ho
Garrett Honke
Percy Liang
Chris Manning
Findings of the Association for Computational Linguistics: NAACL 2025, Association for Computational Linguistics, Albuquerque, New Mexico, pp. 127-152
Preview abstract
Instruction tuning is an important step in making language models useful for direct user interaction. However, the legal domain is underrepresented in typical instruction datasets (e.g., only 10 out of 1600+ tasks in Super-NaturalInstructions). To study whether instruction tuning on legal datasets is necessary for strong legal reasoning, we aggregate 58 annotated legal datasets and write instructions for each, creating LawInstruct. LawInstruct covers 17 global jurisdictions, 24 languages and a total of 12M examples across diverse tasks such as legal QA, summarization of court cases, and legal argument mining. We evaluate our models on LegalBench, measuring legal reasoning across five categories in 162 challenging and realistic legal tasks, and MMLU, to measure potential drops in general reasoning capabilities. We find that legal-specific instruction tuning on Flan-T5 – yielding FLawN-T5 – improves performance on LegalBench across all model sizes, with an aggregate increase of 15 points or 50% over Flan-T5 for the base size. No model size shows performance drops in MMLU. We publish LawInstruct as a resource for further study of instruction tuning in the legal domain.
View details
Preview abstract
One challenge in speech translation is that plenty of spoken content is long-form, but short units are necessary for obtaining high-quality translations. To address this mismatch, we adapt large language models (LLMs) to split long ASR transcripts into segments that can be independently translated so as to maximize the overall translation quality. We overcome the tendency of hallucination in LLMs by incorporating finite-state constraints during decoding; these eliminate invalid outputs without requiring additional training. We discover that LLMs are adaptable to transcripts containing ASR errors through prompt-tuning or fine-tuning. Relative to a state-of-the-art automatic punctuation baseline, our best LLM improves the average BLEU by 2.9 points for English–German, English–Spanish, and English–Arabic TED talk translation in 9 test sets, just by improving segmentation.
View details
UniMorph 3.0: Universal Morphology
Christo Kirov
Matteo Grella
Amrit Nidhi
Patrick Xia
Ekaterina Vylomova
Sabrina J. Mielke
Garrett Nicolai
Miikka Silfverberg
Yuval Pinter
Cassandra L. Jacobs
Ryan Cotterell
Mans Hulden
David Yarowsky
LREC (2020)
Preview abstract
The Universal Morphology (UniMorph) project is a collaborative effort providing morphological information for NLP tasks. The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema. We have implemented several improvements to the extraction pipeline which creates most of our data, so that it is both more complete and more correct. We have added 66 new languages, as well as new parts of speech for 12 languages. We have also amended the schema in several ways. Finally, we present three new community tools: two to validate data for resource creators, and one to make morphological data available from the command line. UniMorph is based at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University in Baltimore, Maryland. This paper details advances made to the schema, tooling, and dissemination of project resources since the UniMorph 2.0 release described at LREC 2018.
View details
Massively Multilingual Pronunciation Mining with WikiPron
Jackson L. Lee
Lucas F. E. Ashby
M. Elizabeth Garza
Yeonju Lee-Sikka
Sean Miller
Alan Wong
2020
Preview abstract
We introduce WikiPron, an open-source command-line tool for extracting pronunciation data from Wiktionary, a free online multilingual dictionary. We first describe the design and use of the library. We then discuss the challenges faced scaling this tool to create an ever-growing database of pronunciations, currently containing 1.7 million pronunciations from 160 languages, both living and dead, natural and constructed. Finally, we validate the pronunciation database by using it to training and evaluating a collection of generic grapheme-to-phoneme models. Software, pronunciation data, and models are all made available under permissive open-source licenses.
View details
Meaning to Form: Measuring Systematicity as Information
Tiago Pimentel
Damian Blasi
Ryan Cotterell
Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) (2019), pp. 1751-1764
Preview abstract
A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an arbitrary relationship between a word form and its meaning, or does some systematic phenomenon pervade? For instance, does the character bigram 'gl' have any systematic relationship to the meaning of words like 'glisten', 'gleam' and 'glow'? In this work, we offer a holistic quantification of the systematicity of the sign using mutual information and recurrent neural networks. We employ these in a data-driven and massively multilingual approach to the question, examining 106 languages. We find a statistically significant reduction in entropy when modeling a word form conditioned on its semantic representation. Encouragingly, we also recover well-attested English examples of systematic affixes. We conclude with the meta-point: Our approximate effect size (measured in bits) is quite small -despite some amount of systematicity between form and meaning, an arbitrary relationship and its resulting benefits dominate human language.
View details
