This page is part of a multi-part series on Model-Agnostic Meta-Learning. If you are already familiar with the topic, use the menu on the right side to jump straight to the part that interests you. Otherwise, we suggest you start at the beginning.
In order to compare the above methods visually, we return to the non-linearized-line-fitting problem from before, see this figure. This time, however, we will plot a single update direction of MAML, FOMAML, Reptile, and iMAML on the combined loss space of the two tasks, such that you can verify whether the methods point into reasonable directions (i.e., towards the local optimum). The combined loss space is defined via the meta loss of the two tasks, i.e., \[\mathcal{L}(\theta) := \sum_{i \in \{0, 1\}} \mathcal{L}_{\tau_i, \text{test}}(\phi_i).\] A word of warning: The update directions are computed on actual data and with the actual algorithms running in your browser on tensorflow.js. If you are experiencing delays on the vector update when moving \(\theta\), you can disable some of the computations via the panel under the figure.
By now, you have a theoretical understanding of the four methods we presented
and might have looked into how the methods produce different updates on the
meta-parameter \( \theta \). To complete the comparison, we want to give
you a short overview of the empirical results of these methods on two
common few-shot benchmarks
| Omniglot | Mini-ImageNet | ||
|---|---|---|---|
| Method | 5-way 1-shot | 20-way 1-shot | 5-way 1-shot |
| MAML | 98.7 ± 0.4% | 95.8 ± 0.3% | 48.70 ± 1.84 % |
| FOMAML | 98.3 ± 0.5% | 89.4 ± 0.5% | 48.07 ± 1.75 % |
| REPTILE | 97.68 ± 0.04% | 89.43 ± 0.14% | 49.97 ± 0.32 % |
| iMAML | 99.16 ± 0.35% | 94.46 ± 0.42% | 48.96 ± 1.84 % |
There is no clear winner. Each method has its place, and only time will show which methods will prevail.
We have now studied some prominent variants of MAML and were able to compare their design, assumptions, and behavior. The next part provides a short conclusion. Also, don't miss out on our further reading section.
Luis Müller implemented the visualization of MAML, FOMAML, Reptile and the Comparision. Max Ploner created the visualization of iMAML and the svelte elements and components. Both wrote the introduction together and contributed most of the text of the other parts. Thomas Goerttler came up with the idea and sketched out the project. He also wrote parts of the manuscript and helped with finalizing the document. Klaus Obermayer provided feedback on the project.
† equal contributors