On first-order meta-learning algorithms
Captured source
source ↗On first-order meta-learning algorithms | OpenAI
March 8, 2018
On first-order meta-learning algorithms
Loading…
Share
Abstract
This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.
Authors
Alex Nichol, Joshua Achiam, John Schulman
Related articles
Scaling laws for reward model overoptimizationPublicationOct 19, 2022
Learning to play Minecraft with Video PreTrainingConclusionJun 23, 2022
Dota 2 with large scale deep reinforcement learningPublicationDec 13, 2019