A Review of Pseudo-Labeling for Computer Vision

Abstract

Deep neural models have achieved state-of-the-art performance on a wide range of problems in computer science, especially in computer vision. However, deep neural networks often require large datasets of labeled samples to generalize effectively. An important area of active research is semi-supervised learning, which attempts to instead utilize large quantities of (easily acquired) unlabeled samples. One family of methods in this space is pseudo-labeling, a class of algorithms that use model outputs to assign labels to unlabeled samples which are then used as labeled samples during training. Such assigned labels, called pseudo-labels, are most commonly associated with the field of semi-supervised learning. In this work, we explore a broader interpretation of pseudo-labels within both self-supervised and unsupervised methods. After a thorough treatment of pseudo-labeling in these areas, we draw the connection between them and identify commonalities between fields, as well as new directions where advancements in one area would likely benefit others, such as curriculum learning and self-supervised regularization.