This page is part of a multi-part series on Model-Agnostic Meta-Learning. If you are already familiar with the topic, use the menu on the right side to jump straight to the part that interests you. Otherwise, we suggest you start at the beginning.
In this section, we explain why MAML is "model-agnostic" and thereby gain a bit more of an overview of the meta-learning field. Metric-based and model-based approaches force constraints on either the sampling (e.g., episodic training) or the model's architecture. MAML, on the other hand, requires only one very general assumption: the model needs to be optimizable by a gradient-based optimizer. Hence, it has been introduced as "model-agnostic". But notice that MAML is still not completely free of assumptions. It is important to view the method in the context of the field to understand what really sets it apart in terms of design, assumptions, and approach, which is what we will consider on this page.
The core idea of metric-based approaches is to compare two samples in a
latent (metric) space: In this space, samples of the same class are supposed
to be close to each other, while two samples from different classes are
supposed to have a large distance (the notion of a distance makes
the latent space a metric space).
Model-based approaches are neural architectures that are deliberately designed
for fast adaption to new tasks without an inclination to overfit.
Memory-Augmented Neural Networks and MetaNet are two examples. Both employ
an external memory while still maintaining the ability to be trained
MAML goes a different route: The neural network is designed the same way
your usual model might be (in the many-shot case). All the magic happens during the
optimization, which is what makes it "optimization-based".
As a consequence, unlike metric-based and model-based approaches, MAML lets
you choose the model architecture freely.
This has the great benefit of being applicable not only to
conventional supervised learning classification tasks but also
to reinforcement learning
In the following figure, you can find a selection of meta-learning methods
learning, their performance on Omniglot, as well as your own accuracy score from the starting page.
Next to recurrent
In the next section, we will take a close look at MAML and study the math behind the method, as well as explore some of the concepts interactively.
Luis Müller fabricated the visualization of MAML, FOMAML, Reptile and the Comparision. Max Ploner created the visualization of iMAML and the svelte elements and components. Both wrote the introduction together and contributed most of the text of the other parts. Thomas Goerttler came up with the idea and sketched out the project. He also wrote parts of the manuscript and helped with finalizing the document. Klaus Obermayer provided feedback on the project.
† equal contributors