Rapid Adaptation with Conditionally Shifted Neurons

About

We describe a mechanism by which artificial neural networks can learn rapid adaptation - the ability to adapt on the fly, with little data, to new tasks - that we call conditionally shifted neurons. We apply this mechanism in the framework of metalearning, where the aim is to replicate some of the flexibility of human learning in machines. Conditionally shifted neurons modify their activation values with task-specific shifts retrieved from a memory module, which is populated rapidly based on limited task experience. On metalearning benchmarks from the vision and language domains, models augmented with conditionally shifted neurons achieve state-of-the-art results.

Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, Adam Trischler · 2017

Related benchmarks

Task	Dataset	Result
Few-shot Image Classification	Mini-Imagenet (test)	Accuracy71.94	235
5-way Classification	miniImageNet (test)	Accuracy71.94	231
Few-shot classification	Mini-ImageNet	1-shot Acc—	178
5-way Few-shot Classification	MiniImagenet	Accuracy (5-shot)71.94	150
5-way Few-shot Classification	Mini-Imagenet (test)	1-shot Accuracy56.88	141
Few-shot classification	miniImageNet standard (test)	5-way 1-shot Acc57.1	138
Few-shot classification	CUB	Accuracy—	104
Image Classification	Mini-Imagenet (test)	Acc (5-shot)71.9	75
5-way Image Classification	MiniImagenet	One-shot Accuracy56.88	67
5-way Classification	miniImageNet 5-way (test)	Accuracy (1-shot)56.88	47