On the Road to Online Adaptation for Semantic Image Segmentation
Riccardo Volpi, Pau de Jorge, Diane Larlus, Gabriela Csurka
News
- May 2022: Pre-trained models and Code released.
- March 2022: Accepted to CVPR 2022, camera ready available on arXiv.
Summary
Our core contribution is introducing the OASIS benchmark, to advance research in Online Adaptation for Semantic Image Segmentation.
“Standard” unsupervised domain adaptation is typically carried out offline, takes into account a limited number of domains and often required human annotation for model validation. These conditions are not well suited for a model released into the wild, which needs to adapt continuously, on the fly and without human intervention. For this reason, we study here the more challenging problem on continual and unsupervised online adaptation and propose an ad hoc benchmark (OASIS). Our benchmark is designed with three well-defined stages: pre-train, validation and deployment: in the first step, the model is pre-trained offline; in the validation phase, the online adaptation method of interest is validated on simulated sequences of temporally correlated frames; finally, in the deployment step the validated approach is released on real sequences recorded into the wild. In the final step, hyper-parameter tuning is no longer allowed. We believe that such scenario better reflects the needs of models released into the wild.
We implement a baseline suite with a broad set of methods at the intersection between unsupervised domain adaptation and continual learning. Furthermore, we propose techniques to overcome catastrophic forgetting. We carry thorough experimental analyses, putting the basis the future research on this topic.
BibTeX
@InProceedings{volpi2022cvpr,
author = {Volpi, Riccardo and de Jorge, Pau and Larlus, Diane and Csurka, Gabriela},
title = {On the Road to Online Adaptation for Semantic Image Segmentation},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2022}
} This work is part of the Lifelong representation learning chair of the MIAI research institute.