# Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates

Source: [Task Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf01>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:10:35.496Z

Authors: Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry

Published: 2020-10-01T13:10:35.000Z

## Abstract

Background: Catastrophic forgetting is the notorious vulnerability of neural networks to the changes in the data distribution during learning. This phenomenon has long been considered a major obstacle for using learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, only a few works consider scenarios in which task boundaries are unknown or not well defined -- task agnostic scenarios. The optimal Bayesian solution for this requires an intractable online Bayes update to the weights posterior. Contributions: We aim to approximate the online Bayes update as accurately as possible. To do so, we derive novel fixed-point equations for the online variational Bayes optimization problem, for multivariate Gaussian parametric distributions. By iterating the posterior through these fixed-point equations, we obtain an algorithm (FOO-VB) for continual learning which can handle non-stationary data distribution using a fixed architecture and without using external memory (i.e. without access to previous data). We demonstrate that our method (FOO-VB) outperforms existing methods in task agnostic scenarios. FOO-VB Pytorch implementation will be available online.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Online Learning on Omniglot-MNIST-FashionMNIST (p = 0.98)](<https://www.sota2.com/research/sota/online-learning-on-omniglot-mnist-fashionmnist-p-0-98.md>): Online Learning; Omniglot-MNIST-FashionMNIST (p = 0.98); Cumulative Online Accuracy: 0.878.

- [Online Learning on Omniglot-MNIST-FashionMNIST p = 0.9](<https://www.sota2.com/research/sota/online-learning-on-omniglot-mnist-fashionmnist-p-0-9.md>): Online Learning; Omniglot-MNIST-FashionMNIST p = 0.9; Cumulative Online Accuracy: 63.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.00373>)

[PDF](<https://arxiv.org/pdf/2010.00373.pdf>)