PUMP: pyramidal and uniqueness matching priors for unsupervised learning of local features

Existing approaches for learning local image descriptors have shown remarkable achievements in a wide range
of geometric tasks. However, most of them require per-pixel correspondence-level supervision, which is difficult to acquire at scale and in high quality. In this paper, we propose to explicitly integrate two matching priors in a single loss in order to learn local descriptors without supervision. Given two images depicting the same scene, we extract pixel descriptors and build a correlation volume. The first prior enforces the local consistency of matches in this volume via a pyramidal structure constructed iteratively at multiple scales using a non-parametric module. The second prior exploits the fact that each descriptor should match with at most one descriptor from the other image. We combine our unsupervised loss with a standard self-supervised loss trained from synthetic image augmentations. Feature descriptors learned by the proposed approach outperform their fully- and self-supervised counterparts on various geometric benchmarks such as visual localization and image matching, achieving state-of-the-art performance

 In these 2 images of the same scene (taken from different points of view), we extract pixel descriptors and build a correlation volume. The colours help to visualize the pixel correspondences. The red dot shows the pixel match once the image has been processed.