Continual learning of visual representations without catastrophic forgetting

In a standard machine learning setup, a model is trained on a bunch of samples until the level of performance is satisfactory or sufficient to be deployed in a real system. In continual learning, the model is retrained over and over again on new data from new domains it encounters throughout its lifetime. There is a classic, decades-old problem related to this retraining known as catastrophic forgetting –  as we provide new information to our model to learn from, it may update at the expense of forgetting what it had previously learned.

Let’s take a concrete example: imagine you have a self-driving vehicle at your disposal which has a pedestrian detection module. The module may have been trained on a large number of pedestrian images, with some bounding boxes around them, to perform detection. You happen to have your own benchmark to evaluate the system and, as you’re generally satisfied with the overall accuracy, you deploy your model.

Alas, after a while, you realise that the model fails when it’s raining hard (perhaps the model doesn’t like umbrellas, who knows…), so you collect more data of ‘under-the-rain pedestrians’ and fine-tune the model. As fall turns to winter you realise that snow is a big issue. You repeat the training process with images of pedestrians in the snow, fine-tune etc. This scenario may be endlessly repeated i.e. every time the model is fine-tuned on new samples from the domain it is unfamiliar with, it starts forgetting the ones it knew before. For example, after having fine-tuned the system on samples of extreme weather, it may forget how to handle pedestrians in the sun.

Learning on several domains (fog, snow etc.) sequentially is the problem we address in our research in continual learning and the work presented at CVPR 2021 [1].  Figure 1 shows how this continual domain adaptation problem differs from standard domain adaptation, domain generalisation or domain randomization.

Approaches to continual learning

The field of continual learning is a very active one in which several strategies have been proposed to address ‘catastrophic forgetting’. We provide an overview of these strategies in the blog post ‘The short memory of artificial neural networks‘. To summarise, memory-based methods cope with the issue of catastrophic forgetting by either storing past samples and enacting regular retraining for past domains, via architecture-based work on the model capacity and structure in general (for example, by having task-specific sub-networks) or, for regularisation-based methods, by improving the objective that the network optimises. Our method is of the third flavour whereby we propose a regulariser which builds on recent advances in meta-learning research. We also show that it can work side-by-side with memory-based approaches, which are often the most effective.

As these approaches are so effective, the question naturally arises of why one wouldn’t simply start from a memory-based approach in the first place. One of the reasons for not doing so is is that it’s often not possible to store data from every domain (i.e. for reasons of privacy) or because a good learning system that does not require replaying would avoid re-doing the same computation over and over again. We also believe that learning how not to forget without explicitly looking at the past is a very interesting research direction per se.

Our approach

Going back to the self-driving car example we looked at earlier, we had the intuition that, if we can improve the robustness of our model in the first place with domain randomization, future fine-tuning sessions will generally be lighter. If adaptation is lighter, forgetting should be less severe. Intuitively, we can expect that, if a model is already resilient to some extent to fog, the fine-tuning process on foggy examples will alter it less and it will forget less.

Yet, in general we don’t know in advance where the model will be released and the conditions it will face before it’s too late. Preparing for the unknown, which constitutes the real world, is a problem researchers face all the time when they train their models on simulators. To address this, a seminal paper of Tobin et al. (2) proposed a technique called “domain randomization” whereby, if we heavily randomize (e.g. alter with random transformations) the aspect of our training samples, we increase the robustness of the agent performance in the real-world.

Even if we ourselves don’t train on simulation, we share the same goal. Working with images, it’s pretty straightforward to generate a large variety of transformed copies of each training sample, thanks to the combination of simple image transformations (“augment the brightness, then reduce the contrast, then flip the image” etc.) also known as data augmentation. Figure 2 illustrates how, by applying some image transformations, it’s possible to approximate sunset or overcast conditions.

So, the first contribution of this work is showing that, indeed, domain randomization can help learning more robust models for continual domain adaptation problems – that is, models that forget less when transferred to new domains. It is extremely simple to apply and can essentially be part of any continual learning algorithm.

So, an initial contribution of our work shows that domain randomization can help in learning more robust models for continual domain adaptation problems so that models forget less when transferred to new domains. It’s extremely simple to apply and can essentially be part of any continual learning algorithm.

Armed with a strong baseline, we then wondered if there was a more principled and effective way of exploiting the image transformations. After all, with vanilla domain randomization, we’re not really instructing our models to forget less –  it’s simply a positive side-effect of learning more general visual representations using more diverse training samples. One thing that came to mind is that these sets of transformed images obtained from the same image transformations could be seen as “transformed domains” or “meta-domains” in meta-learning terminology (see Figure 1).

Intuitively, we use the transformed training images as a proxy for future domains. To illustrate this, let’s go back to the vehicle pedestrian detection module. Randomly creating image transformations will create at least some meta-domains that are a good proxy for foggy/cloudy scenarios, even if we start from clean weather samples. Running “meta-learning” steps towards such meta-domains emulates the future fine-tuning stages that the model will need to undergo. This is how we exploit image transformations. Inspired by the Model-Agnostic Meta-Learning (MAML) algorithm of Finn et al. [3], we simulate fine-tuning steps in their favour and optimise for good adaptation performance and for good performance on retaining the current domain samples.

We can perform this meta-learning operation at every fine-tuning step of the sequence of encountered domains, as summarised in Figure 3.

More details on the approach are described in the paper (1).

Experimental validation

We study the performance of our proposed method on standard computer vision tasks such as digit recognition, object recognition and urban scene semantic segmentation. For each experiment, we consider a sequence of domains and different adaptation stages – one for each domain shift. For each adaptation stage, we fine-tune our model on the new domain.

Our results on the PACS dataset [4] are shown in Figure 4. We train on images from the sketch domain, then move to cartoons, paintings and photos – and at the end evaluate on all the domains.

We can see a steady increase in performance when applying domain randomization, and a further boost when also applying our meta-learning approach (Figure 4).

You’ll find more experiments and ablation studies in our paper [1], in particular when a small memory storing samples from previous domains is available.

Summary and future work

We propose a new approach to tackle continual domain adaptation where we exploit the effectiveness of domain randomization to learn more robust visual representations and show that this helps in mitigating catastrophic forgetting. Building on this, we use image transformations to build meta-domains and show that a meta-learning strategy leads to representations that are inherently more robust.

For future research, it would be interesting to understand if more principled approaches to craft these meta-domains could be adopted as in this work, we limit ourselves to simple data augmentation rules, but believe that more sophisticated approaches could be adopted.

References:

[1] Continual adaptation of visual representations via domain randomization and meta-learning. R. Volpi, D. Larlus, G. Rogez, International Conference ​on Vision and Pattern Recognition (CVPR), virtual, 19–25 June, 2021.

[2] Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World, J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, P. Abbeel, International Conference on Intelligent Robots and Systems (IROS), 24-28 September, Vancouver, Canada, 2017.

[3] Model-Agnostic Meta-learning for Fast Adaptation of Deep Networks, C Finn, P. Abbeel, S. Levine, Proceedings of the 34^th International Conference on Machine Learning (ICML), Volume 70 pp.1126-1135, August 2017

[4] Deeper, Broader and Artier Domain Generalization, D. Li, Y. Yang, Y-Z. Song, T. Hospedales, conference on vision and Pattern Recognition (CVPR), June, Hawaii, USA, 2017