Adversarial Nibbler: A DataPerf Challenge for Text-to-Image Models
Abstract
Machine learning progress has been strongly influenced by the data used for
model training and evaluation. Only recently however, have development teams
shifted their focus more to the data. This shift has been triggered by the numerous
reports about biases and errors discovered in AI datasets. Thus, the data-centric
AI movement introduced the notion of iterating on the data used in AI systems, as
opposed to the traditional model-centric AI approach, which typically treats the
data as a given static artifact in model development. With the recent advancement of
generative AI, the role of data is even more crucial for successfully developing more
factual and safe models. DataPerf challenges follow up on recent successful data-
centric challenges drawing attention to the data used for training and evaluation of
machine learning model. Specifically, Adversarial Nibbler focuses on data used for
safety evaluation of generative text-to-image models. A typical bottleneck in safety
evaluation is achieving a representative diversity and coverage of different types
of examples in the evaluation set. Our competition aims to gather a wide range
of long-tail and unexpected failure modes for text-to-image models in order to
identify as many new problems as possible and use various automated approaches
to expand the dataset to be useful for training, fine tuning, and evaluation.
model training and evaluation. Only recently however, have development teams
shifted their focus more to the data. This shift has been triggered by the numerous
reports about biases and errors discovered in AI datasets. Thus, the data-centric
AI movement introduced the notion of iterating on the data used in AI systems, as
opposed to the traditional model-centric AI approach, which typically treats the
data as a given static artifact in model development. With the recent advancement of
generative AI, the role of data is even more crucial for successfully developing more
factual and safe models. DataPerf challenges follow up on recent successful data-
centric challenges drawing attention to the data used for training and evaluation of
machine learning model. Specifically, Adversarial Nibbler focuses on data used for
safety evaluation of generative text-to-image models. A typical bottleneck in safety
evaluation is achieving a representative diversity and coverage of different types
of examples in the evaluation set. Our competition aims to gather a wide range
of long-tail and unexpected failure modes for text-to-image models in order to
identify as many new problems as possible and use various automated approaches
to expand the dataset to be useful for training, fine tuning, and evaluation.
Research Areas
