Trimmed Maximum Likelihood Estimation for Robust Learning in Generalized Linear Models

Abhimanyu Das
Pranjal Awasthi
Weihao Kong
NeurIPS 2022 (2022)

Abstract

We study the problem of learning generalized linear models under adversarial corruptions.
We analyze a classical heuristic called the \textit{iterative trimmed maximum likelihood estimator} which is known to be effective against \textit{label corruptions} in practice. Under label corruptions, we prove that this simple estimator achieves minimax near-optimal risk on a wide range of generalized linear models, including Gaussian regression, Poisson regression and Binomial regression. Finally, we extend the estimator to the much more challenging setting of \textit{label and covariate corruptions} and demonstrate its robustness and optimality in that setting as well.

Research Areas