Active Learning with Expected Error Reduction
In collaboration with Stanford University, University of Washington
AuthorsSteve Mussmann, Julia Reisler, Daniel Tsai, Ehsan Mousavi, Shayne O’Brien, Moises Goldszmidt
Active Learning with Expected Error Reduction
In collaboration with Stanford University, University of Washington
AuthorsSteve Mussmann, Julia Reisler, Daniel Tsai, Ehsan Mousavi, Shayne O’Brien, Moises Goldszmidt
Active learning has been studied extensively as a method for efficient data col- lection. Among the many approaches in literature, Expected Error Reduction (EER) Roy & McCallum (2001) has been shown to be an effective method for ac- tive learning: select the candidate sample that, in expectation, maximally decreases the error on an unlabeled set. However, EER requires the model to be retrained for every candidate sample and thus has not been widely used for modern deep neural networks due to this large computational cost. In this paper we reformulate EER under the lens of Bayesian active learning and derive a computationally efficient version that can use any Bayesian parameter sampling method (such as Gal & Ghahramani (2016)). We then compare the empirical performance of our method using Monte Carlo dropout for parameter sampling against state of the art methods in the deep active learning literature. Experiments are performed on four standard benchmark datasets and three WILDS datasets (Koh et al., 2021). The results indicate that our method outperforms all other methods except one in the data shift scenario – a model dependent, non-information theoretic method that requires an order of magnitude higher computational cost (Ash et al., 2019).
Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation
October 21, 2026research area Computer Vision, research area Data Science and Annotationconference NeurIPS
While foundation models and vision transformers (ViTs) have redefined visual representations, current active learning (AL) strategies for these backbones largely operate at the patch or region level, where annotation requirements remain high due to extensive pixel labeling. Extending ViT-based AL to the pixel-level, low-budget regime is uniquely challenging: with as little as one pixel query per image, a method must simultaneously identify the…
Empirical Evaluation of Active Learning Techniques for Neural MT
November 3, 2019research area Speech and Natural Language Processingconference EMNLP
Active learning (AL) for machine translation (MT) has been well-studied for the phrase-based MT paradigm. Several AL algorithms for data sampling have been proposed over the years. However, given the rapid advancement in neural methods, these algorithms have not been thoroughly investigated in the context of neural MT (NMT). In this work, we address this missing aspect by conducting a systematic comparison of different AL methods in a simulated…