On the Road to Online Adaptation for Semantic Image Segmentation

CVPR 2022

Riccardo Volpi, Pau de JorgeDiane Larlus, Gabriela Csurka

 

This video shows how the NAVER LABS Europe continual learning online adaptation method (right), applied to image segmentation (classes listed in boxes), compares to image segmentation without adaptation (left) or with a Naive continual learning method (middle) which suffers from catastrophic forgetting (classes forgotten over time).

blank

News

Summary

Our core contribution is introducing the OASIS benchmark, to advance research in Online Adaptation for Semantic Image Segmentation.

“Standard” unsupervised domain adaptation is typically carried out offline, takes into account a limited number of domains and often required human annotation for model validation. These conditions are not well suited for a model released into the wild, which needs to adapt continuously, on the fly and without human intervention. For this reason, we study here the more challenging problem on continual and unsupervised online adaptation and propose an ad hoc benchmark (OASIS). Our benchmark is designed with three well-defined stages: pre-train, validation and deployment: in the first step, the model is pre-trained offline; in the validation phase, the online adaptation method of interest is validated on simulated sequences of temporally correlated frames; finally, in the deployment step the validated approach is released on real sequences recorded into the wild. In the final step, hyper-parameter tuning is no longer allowed. We believe that such scenario better reflects the needs of models released into the wild.

We implement a baseline suite with a broad set of methods at the intersection between unsupervised domain adaptation and continual learning. Furthermore, we propose techniques to overcome catastrophic forgetting. We carry thorough experimental analyses, putting the basis the future research on this topic.

BibTeX

@InProceedings{volpi2022cvpr,
  author = {Volpi, Riccardo and de Jorge, Pau and Larlus, Diane and Csurka, Gabriela},
  title = {On the Road to Online Adaptation for Semantic Image Segmentation},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2022}
}

This work is part of the Lifelong representation learning chair of the MIAI research institute.

This web site uses cookies for the site search, to display videos and for aggregate site analytics.

Learn more about these cookies in our privacy notice.

blank

Cookie settings

You may choose which kind of cookies you allow when visiting this website. Click on "Save cookie settings" to apply your choice.

FunctionalThis website uses functional cookies which are required for the search function to work and to apply for jobs and internships.

AnalyticalOur website uses analytical cookies to make it possible to analyse our website and optimize its usability.

Social mediaOur website places social media cookies to show YouTube and Vimeo videos. Cookies placed by these sites may track your personal data.

blank