Continual Learning Ai | AI Final Year Project Report

Complete Technical Paper Content — Continual Learning Ai

1. Executive Abstract, Problem Formulation & Background

In computer science, incremental learning is a method of machine learning in which input data is continuously used to extend the existing model's knowledge i.e. to further train the model. It represents a dynamic technique of supervised learning and unsupervised learning that can be applied when training data becomes available gradually over time or its size is out of system memory limits. Algorithms that can facilitate incremental learning are known as incremental machine learning algorithms. In contemporary machine learning literature, settings that require learning from a sequence of tasks while limiting catastrophic forgetting are often discussed under the closely related term continual learning. Many traditional machine learning algorithms inherently support incremental learning. Other algorithms can be adapted to facilitate incremental learning. Examples of incremental algorithms include decision trees (IDE4, ID5R and gaenari), decision rules, artificial neural networks (RBF networks, Learn++, Fuzzy ARTMAP, TopoART, and IGNG) or the incremental SVM. The aim of incremental learning is for the learning model to adapt to new data without forgetting its existing knowledge. Some incremental learners have built-in some parameter or assumption that controls the relevancy of old data, while others, called stable incremental machine learning algorithms, learn representations of the training data that are not even partially forgotten over time. Fuzzy ART and TopoART are two examples for this second approach. Incremental algorithms are frequently applied to data streams or big data, addressing issues in data availability and resource scarcity respectively. Stock trend prediction and user profiling are some examples of data streams where new data becomes continuously available. Applying incremental learning to big data aims to produce faster classification or forecasting times.

In deep learning, fine-tuning is the process of adapting a computational model trained for one task (the upstream task) to perform a different, usually more specific, task (the downstream task). It is considered a form of transfer learning, as it reuses knowledge learned from the original training objective. Fine-tuning involves applying additional training (e.g., on new data) to the parameters of a neural network that have been pre-trained. Many variants exist. The additional training can be applied to the entire neural network, or to only a subset of its layers, in which case the layers that are not being fine-tuned are "frozen" (i.e., not changed during backpropagation). A model may also be augmented with "adapters"—lightweight modules inserted into the model's architecture that nudge the embedding space for domain adaptation. These contain far fewer parameters than the original model and can be fine-tuned in a parameter-efficient way by tuning only their weights and leaving the rest of the model's weights frozen. For some architectures, such as convolutional neural networks, it is common to keep the earlier layers (those closest to the input layer) frozen, as they capture lower-level features, while later layers often discern high-level features that can be more related to the task that the model is trained on. Models that are pre-trained on large, general corpora are usually fine-tuned by reusing their parameters as a starting point and adding a task-specific layer trained from scratch. Fine-tuning the full model is also common and often yields better results, but is more computationally expensive. Fine-tuning is typically accomplished via supervised learning, but there are also techniques to fine-tune a model using weak supervision. Fine-tuning can be combined with a reinforcement learning from human feedback-based objective to produce language models such as ChatGPT (a fine-tuned version of GPT models) and Sparrow.

Avalanche is an End-to-End Continual Learning Library based on PyTorch , born within ContinualAI with the goal of providing a shared and collaborative open-source (MIT licensed) codebase for fast prototyping , training and reproducible evaluation of continual learning algorithms.

Looking for continual learning baselines ? In the CL-Baseline sibling project based on Avalanche we reproduce seminal papers results you can directly use in your experiments!

Benchmarks : This module maintains a uniform API for data handling: mostly generating a stream of data from one or more datasets. It contains all the major CL benchmarks (similar to what has been done for torchvision ).

Training : This module provides all the necessary utilities concerning model training. This includes simple and efficient ways of implement new continual learning strategies as well as a set pre-implemented CL baselines and state-of-the-art algorithms you will be able to use for comparison!

Evaluation : This modules provides all the utilities and metrics that can help evaluate a CL algorithm with respect to all the factors we believe to be important for a continually learning system.

Models : In this module you'll be able to find several model architectures and pre-trained models that can be used for your continual learning experiment (similar to what has been done in torchvision.models ).

Logging : It includes advanced logging and plotting features, including native stdout , file and TensorBoard support (How cool it is to have a complete, interactive dashboard, tracking your experiment metrics in real-time with a single line of code?)

Avalanche the first experiment of a End-to-end Library for reproducible continual learning research & development where you can find benchmarks , algorithms, evaluation metrics and much more, in the same place .

We know that learning a new tool may be tough at first . This is why we made Avalanche as easy as possible to learn with a set of resources that will help you along the way.

For example, you may start with our 5-minutes guide that will let you acquire the basics about Avalanche and how you can use it in your research project:

We have also prepared for you a large set of examples & snippets you can plug-in directly into your code and play with:

Having completed these two sections, you will already feel with superpowers ⚡, this is why we have also created an in-depth tutorial that will cover all the aspect of Avalanche in details and make you a true Continual Learner ! 👨‍🎓️

2. Related Work, Literature Survey & Algorithmic Foundations

If you use Avalanche in your research project, please remember to cite our JMLR-MLOSS paper https://jmlr.org/papers/v24/23-0130.html . This will help us make Avalanche better known in the machine learning community, ultimately making a better tool for everyone:

you can also cite the previous CLVision @ CVPR2021 workshop paper: "Avalanche: an End-to-End Library for Continual Learning" .

Avalanche is the flagship open-source collaborative project of ContinualAI : a non profit research organization and the largest open community on Continual Learning for AI.

Do you have a question, do you want to report an issue or simply ask for a new feature? Check out the Questions & Issues center. Do you want to improve Avalanche yourself? Follow these simple rules on How to Contribute .

The Avalanche project is maintained by the collaborative research team ContinualAI Lab and used extensively by the Units of the ContinualAI Research (CLAIR) consortium, a research network of the major continual learning stakeholders around the world.

We are always looking for new awesome members willing to join the ContinualAI Lab , so check out our official website if you want to learn more about us and our activities, or contact us .

import torch from torch.nn import CrossEntropyLoss from torch.optim import SGD from avalanche.benchmarks.classic import PermutedMNIST from avalanche.training.plugins import EvaluationPlugin from avalanche.evaluation.metrics import accuracy_metrics from avalanche.models import SimpleMLP from avalanche.training.supervised import Naive # Config device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") # model model = SimpleMLP(num_classes=10) # CL Benchmark Creation perm_mnist = PermutedMNIST(n_experiences=3) train_stream = perm_mnist.train_stream test_stream = perm_mnist.test_stream # Prepare for training & testing optimizer = SGD(model.parameters(), lr=0.001, momentum=0.9) criterion = CrossEntropyLoss() eval_plugin = EvaluationPlugin( accuracy_metrics(minibatch=True, epoch=True, epoch_running=True, experience=True, stream=True)) # Continual learning strategy cl_strategy = Naive( model, optimizer, criterion, train_mb_size=32, train_epochs=2, eval_mb_size=32, evaluator=eval_plugin, device=device) # train and test loop results = [] for train_task in train_stream: cl_strategy.train(train_task, num_workers=4) results.append(cl_strategy.eval(test_stream))

import torch import torch.nn as nn from torch.nn import CrossEntropyLoss from torch.optim import SGD from torchvision import transforms from torchvision.datasets import MNIST from torchvision.transforms import ToTensor, RandomCrop from torch.utils.data import DataLoader import numpy as np from copy import copy # Config device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") # model class SimpleMLP(nn.Module): def __init__(self, num_classes=10, input_size=28*28): super(SimpleMLP, self).__init__() self.features = nn.Sequential( nn.Linear(input_size, 512), nn.ReLU(inplace=True), nn.Dropout(), ) self.classifier = nn.Linear(512, num_classes) self._input_size = input_size def forward(self, x): x = x.contiguous() x = x.view(x.size(0), self._input_size) x = self.features(x) x = self.classifier(x) return x model = SimpleMLP(num_classes=10) # CL Benchmark Creation list_train_dataset = [] list_test_dataset = [] rng_permute = np.random.RandomState(0) train_transform = transforms.Compose([ RandomCrop(28, padding=4), ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) test_transform = transforms.Compose([ ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ]) # permutation transformation class PixelsPermutation(object): def __init__(self, index_permutation): self.permutation = index_permutation def __call__(self, x): return x.view(-1)[self.permutation].view(1, 28, 28) def get_permutation(): return torch.from_numpy(rng_permute.permutation(784)).type(torch.int64) # for every incremental step permutations = [] for i in range(3): # choose a random permutation of the pixels in the image idx_permute = get_permutation() current_perm = PixelsPermutation(idx_permute) permutations.append(idx_permute) # add the permutation to the default dataset transformation train_transform_list = train_transform.transforms.copy() train_transform_list.append(current_perm) new_train_transform = transforms.Compose(train_transform_list) test_transform_list = test_transform.transforms.copy() test_transform_list.append(current_perm) new_test_transform = transforms.Compose(test_transform_list) # get the datasets with the constructed transformation permuted_train = MNIST(root='./data/mnist', download=True, transform=new_train_transform) permuted_test = MNIST(root='./data/mnist', train=False, download=True, transform=new_test_transform) list_train_dataset.append(permuted_train) list_test_dataset.append(permuted_test) # Train optimizer = SGD(model.parameters(), lr=0.001, momentum=0.9) criterion = CrossEntropyLoss() for task_id, train_dataset in enumerate(list_train_dataset): train_data_loader = DataLoader( train_dataset, num_workers=4, batch_size=32) for ep in range(2): for iteration, (train_mb_x, train_mb_y) in enumerate(train_data_loader): optimizer.zero_grad() train_mb_x = train_mb_x.to(device) train_mb_y = train_mb_y.to(device) # Forward logits = model(train_mb_x) # Loss loss = criterion(logits, train_mb_y) # Backward loss.backward() # Update optimizer.step() # Test acc_results = [] for task_id, test_dataset in enumerate(list_test_dataset): test_data_loader = DataLoader( test_dataset, num_workers=4, batch_size=32) correct = 0 for iteration, (test_mb_x, test_mb_y) in enumerate(test_data_loader): # Move mini-batch data to device test_mb_x = test_mb_x.to(device) test_mb_y = test_mb_y.to(device) # Forward test_logits = model(test_mb_x) # Loss test_loss = criterion(test_logits, test_mb_y) # compute acc correct += test_mb_y.eq(test_logits.argmax(dim=1)).sum().item() acc_results.append(correct / len(test_dataset))

@article{JMLR:v24:23-0130, author = {Antonio Carta and Lorenzo Pellegrini and Andrea Cossu and Hamed Hemati and Vincenzo Lomonaco}, title = {Avalanche: A PyTorch Library for Deep Continual Learning}, journal = {Journal of Machine Learning Research}, year = {2023}, volume = {24}, number = {363}, pages = {1--6}, url = {http://jmlr.org/papers/v24/23-0130.html} }

@InProceedings{lomonaco2021avalanche, title={Avalanche: an End-to-End Library for Continual Learning}, author={Vincenzo Lomonaco and Lorenzo Pellegrini and Andrea Cossu and Antonio Carta and Gabriele Graffieti and Tyler L. Hayes and Matthias De Lange and Marc Masana and Jary Pomponi and Gido van de Ven and Martin Mundt and Qi She and Keiland Cooper and Jeremy Forest and Eden Belouadah and Simone Calderara and German I. Parisi and Fabio Cuzzolin and Andreas Tolias and Simone Scardapane and Luca Antiga and Subutai Amhad and Adrian Popescu and Christopher Kanan and Joost van de Weijer and Tinne Tuytelaars and Davide Bacciu and Davide Maltoni}, booktitle={Proceedings of IEEE Conference on Computer Vision and Pattern Recognition}, series={2nd Continual Learning in Computer Vision Workshop}, year={2021} }

Benchmarks : Machine learning researchers need multiple benchmarks with efficient data handling utils to design and prototype new algorithms. Quantitative results on ever-changing benchmarks has been one of the driving forces of Deep Learning .

Training : Efficient implementation and training of continual learning algorithms; comparisons with other baselines and state-of-the-art methods become fundamental to asses the quality of an original algorithmic proposal.

Evaluation : Training utils and Benchmarks are not enough alone to push continual learning research forward. Comprehensive and sound evaluation protocols and metrics need to be employed as well.

With Avalanche, you can find all these three fundamental pieces together and much more, in a single and coherent, well-maintained codebase.

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

Let's take a quick tour on how you can use Avalanche for your research projects with a 5-minutes guide , for researchers on the run !

Let's first install Avalanche . Please, check out our How to Install guide for further details.

Evaluation : This modules provides all the utilities and metrics that can help in evaluating a CL algorithm with respect to all the factors we believe to be important for a continually learning system.

Logging : It includes advanced logging and plotting features, including native stdout, file and Tensorboard support (How cool it is to have a complete, interactive dashboard, tracking your experiment metrics in real-time with a single line of code?)

In the graphic below, you can see how Avalanche sub-modules are available and organized as well:

We will learn more about each of them during this tutorial series, but keep in mind that the [Avalanche API documentation](https://avalanche-api.continualai.org/en/latest/) is your friend as well!

Datasets : a comprehensive list of PyTorch Datasets ready to use (It includes all the Torchvision Datasets and more!).

Classic Benchmarks : a set of classic Continual Learning Benchmarks ready to be used (there can be multiple benchmarks based on a single dataset).

Generators : a set of functions you can use to generate your own benchmark starting from any PyTorch Dataset!

The Avalanche benchmarks (instances of the Scenario class), contains several attributes that describe the benchmark. However, the most important ones are the train and test streams.

In Avalanche we often suppose to have access to these two parallel stream of data (even though some benchmarks may not provide such feature, but contain just a unique test set).

4. Neural Network Backbone, Layer Specifications & Loss Formulations

Each of these streams are iterable, indexable and sliceable objects that are composed of experiences . Experiences are batch of data (or "tasks") that can be provided with or without a specific task label.

Avalanche maintains a set of commonly used benchmarks built on top of one or multiple datasets.

What if we want to create a new benchmark that is not present in the "Classic" ones? Well, in that case Avalanche offers a number of utilities that you can use to create your own benchmark with maximum flexibility: the benchmark generators !

The specific scenario generators are useful when starting from one or multiple PyTorch datasets and you want to create a "New Instances" or "New Classes" benchmark: i.e. it supports the easy and flexible creation of a Domain-Incremental , Class-Incremental or Task-Incremental scenarios among others.

Finally, if your ideal benchmark does not fit well in the aforementioned Domain-Incremental , Class-Incremental or Task-Incremental scenarios, you can always use our generic generators :

provide a set of standard continual learning baselines that can be easily run for comparison;

provide the necessary utilities to implement and run your own strategy in the most efficient and simple way possible thanks to the building blocks we already prepared for you.

If you want to compare your strategy with other classic continual learning algorithms or baselines, in Avalanche this is as simple as creating an object:

The simplest way to build your own strategy is to create a python class that implements the main train and eval methods.

While this is the easiest possible way to add your own strategy, Avalanche supports more sophisticated modalities (based on callbacks ) that lets you write more neat, modular and reusable code , inheriting functionality from a parent classes and using pre-implemented plugins .

Check out more details about what Avalanche can offer in this module following the " Training " chapter of the "From Zero to Hero" tutorial!

The evaluation module is quite straightforward: it offers all the basic functionalities to evaluate and keep track of a continual learning experiment.

This is mostly done through the Metrics and the Loggers . The Metrics provide a set of classes which implements the main continual learning metrics like Accuracy, Forgetting, Memory Usage, Running Times, etc. Metrics should be created via the utility functions (e.g. accuracy_metrics , timing_metrics and others) specifying in the arguments when those metrics should be computed (after each minibatch, epoch, experience etc...). The Loggers specify a way to report the metrics (e.g. with Tensorboard, on console or others). Loggers are created by instantiating the respective class.

Metrics and loggers interact via the Evaluation Plugin : this is the main object responsible of tracking the experiment progress. Metrics and loggers are directly passed to the EvaluationPlugin instance. You will see the output of the loggers automatically during training and evaluation! Let's see how to put this together in few lines of code:

5. Training Protocol, Convergence Behavior & Experimental Evaluation

For more details about the evaluation module (how to write new metrics/loggers, a deeper tutorial on metrics) check out the extended guide in the "Evaluation" chapter of the "From Zero to Hero" Avalanche tutorial!

You've learned how to install Avalanche , how to create benchmarks that can suit your needs, how you can create your own continual learning algorithm and how you can evaluate its performance.

Here we show how you can use all these modules together to design your experiments as quantitative supporting evidence for your research project or paper.

Avalanche ├── Benchmarks │ ├── Classic │ ├── Datasets │ ├── Generators │ ├── Scenarios │ └── Utils ├── Evaluation │ ├── Metrics | └── Utils ├── Training │ ├── Strategies │ ├── Plugins | └── Utils ├── Models └── Loggers

from avalanche.benchmarks.datasets import MNIST, FashionMNIST, KMNIST, EMNIST, \ QMNIST, FakeData, CocoCaptions, CocoDetection, LSUN, ImageNet, CIFAR10, \ CIFAR100, STL10, SVHN, PhotoTour, SBU, Flickr8k, Flickr30k, VOCDetection, \ VOCSegmentation, Cityscapes, SBDataset, USPS, HMDB51, UCF101, CelebA, \ CORe50Dataset, TinyImagenet, CUB200, OpenLORIS, MiniImageNetDataset, \ Stream51, CLEARDataset

from avalanche.benchmarks.classic import CORe50, SplitTinyImageNet, SplitCIFAR10, \ SplitCIFAR100, SplitCIFAR110, SplitMNIST, RotatedMNIST, PermutedMNIST, SplitCUB200 # creating the benchmark (scenario object) perm_mnist = PermutedMNIST( n_experiences=3, seed=1234, ) # recovering the train and test streams train_stream = perm_mnist.train_stream test_stream = perm_mnist.test_stream # iterating over the train stream for experience in train_stream: print("Start of task ", experience.task_label) print('Classes in this task:', experience.classes_in_this_experience) # The current Pytorch training set can be easily recovered through the # experience current_training_set = experience.dataset # ...as well as the task_label print('Task {}'.format(experience.task_label)) print('This task contains', len(current_training_set), 'training examples') # we can recover the corresponding test experience in the test stream current_test_set = test_stream[experience.current_experience].dataset print('This task contains', len(current_test_set), 'test examples')

from avalanche.benchmarks.generators import nc_benchmark, ni_benchmark from torchvision.datasets import MNIST mnist_train = MNIST('.', train=True, download=True) mnist_test = MNIST('.', train=False) benchmark = ni_benchmark( mnist_train, mnist_test, n_experiences=10, shuffle=True, seed=1234, balance_experiences=True ) benchmark = nc_benchmark( mnist_train, mnist_test, n_experiences=10, shuffle=True, seed=1234, task_labels=False )

from avalanche.benchmarks.generators import filelist_benchmark, dataset_benchmark, \ tensors_benchmark, paths_benchmark

from avalanche.models import SimpleMLP from avalanche.training import Naive, CWRStar, Replay, GDumb, \ Cumulative, LwF, GEM, AGEM, EWC, AR1 from torch.optim import SGD from torch.nn import CrossEntropyLoss model = SimpleMLP(num_classes=10) cl_strategy = Naive( model, SGD(model.parameters(), lr=0.001, momentum=0.9), CrossEntropyLoss(), train_mb_size=100, train_epochs=4, eval_mb_size=100 )

from torch.utils.data import DataLoader class MyStrategy(): """My Basic Strategy""" def __init__(self, model, optimizer, criterion): self.model = model self.optimizer = optimizer self.criterion = criterion def train(self, experience): # here you can implement your own training loop for each experience (i.e. # batch or task). train_dataset = experience.dataset t = experience.task_label train_data_loader = DataLoader( train_dataset, num_workers=4, batch_size=128 ) for epoch in range(1): for mb in train_data_loader: # you magin here... pass def eval(self, experience): # here you can implement your own eval loop for each experience (i.e. # batch or task). eval_dataset = experience.dataset t = experience.task_label eval_data_loader = DataLoader( eval_dataset, num_workers=4, batch_size=128 ) # eval here

from avalanche.models import SimpleMLP from avalanche.benchmarks import SplitMNIST # Benchmark creation benchmark = SplitMNIST(n_experiences=5) # Model Creation model = SimpleMLP(num_classes=benchmark.n_classes) # Create the Strategy Instance (MyStrategy) cl_strategy = MyStrategy( model, SGD(model.parameters(), lr=0.001, momentum=0.9), CrossEntropyLoss()) # Training Loop print('Starting experiment...') for exp_id, experience in enumerate(benchmark.train_stream): print("Start of experience ", experience.current_experience) cl_strategy.train(experience) print('Training completed') print('Computing accuracy on the current test set') cl_strategy.eval(benchmark.test_stream[exp_id])

# utility functions to create plugin metrics from avalanche.evaluation.metrics import accuracy_metrics, loss_metrics, forgetting_metrics from avalanche.logging import InteractiveLogger, TensorboardLogger from avalanche.training.plugins import EvaluationPlugin eval_plugin = EvaluationPlugin( # accuracy after each training epoch # and after each evaluation experience accuracy_metrics(epoch=True, experience=True), # loss after each training minibatch and each # evaluation stream loss_metrics(minibatch=True, stream=True), # catastrophic forgetting after each evaluation # experience forgetting_metrics(experience=True, stream=True), # add as many metrics as you like loggers=[InteractiveLogger(), TensorboardLogger()]) # pass the evaluation plugin instance to the strategy # strategy = EWC(..., evaluator=eval_plugin) # THAT'S IT!!

from avalanche.benchmarks.classic import SplitMNIST from avalanche.evaluation.metrics import forgetting_metrics, accuracy_metrics,\ loss_metrics, timing_metrics, cpu_usage_metrics, StreamConfusionMatrix,\ disk_usage_metrics, gpu_usage_metrics from avalanche.models import SimpleMLP from avalanche.logging import InteractiveLogger, TextLogger, TensorboardLogger from avalanche.training.plugins import EvaluationPlugin from avalanche.training import Naive from torch.optim import SGD from torch.nn import CrossEntropyLoss benchmark = SplitMNIST(n_experiences=5) # MODEL CREATION model = SimpleMLP(num_classes=benchmark.n_classes) # DEFINE THE EVALUATION PLUGIN and LOGGERS # The evaluation plugin manages the metrics computation. # It takes as argument a list of metrics, collectes their results and returns # them to the strategy it is attached to. # log to Tensorboard tb_logger = TensorboardLogger() # log to text file text_logger = TextLogger(open('log.txt', 'a')) # print to stdout interactive_logger = InteractiveLogger() eval_plugin = EvaluationPlugin( accuracy_metrics(minibatch=True, epoch=True, experience=True, stream=True), loss_metrics(minibatch=True, epoch=True, experience=True, stream=True), timing_metrics(epoch=True), cpu_usage_metrics(experience=True), forgetting_metrics(experience=True, stream=True), StreamConfusionMatrix(num_classes=benchmark.n_classes, save_image=False), disk_usage_metrics(minibatch=True, epoch=True, experience=True, stream=True), loggers=[interactive_logger, text_logger, tb_logger] ) # CREATE THE STRATEGY INSTANCE (NAIVE) cl_strategy = Naive( model, SGD(model.parameters(), lr=0.001, momentum=0.9), CrossEntropyLoss(), train_mb_size=500, train_epochs=1, eval_mb_size=100, evaluator=eval_plugin) # TRAINING LOOP print('Starting experiment...') results = [] for experience in benchmark.train_stream: print("Start of experience: ", experience.current_experience) print("Current Classes: ", experience.classes_in_this_experience) # train returns a dictionary which contains all the metric values res = cl_strategy.train(experience, num_workers=4) print('Training completed') print('Computing accuracy on the whole test set') # eval also returns a dictionary which contains all the metric values results.append(cl_strategy.eval(benchmark.test_stream, num_workers=4))

Avalanche is an open-source, end-to-end continual learning library developed by ContinualAI to accelerate research and development in continual learning. Built on PyTorch, Avalanche provides a comprehensive framework for prototyping, training, and evaluating continual learning algorithms, making it easier for researchers and practitioners to advance their work in the field.