Tutorial

A Tutorial on Meta-Reinforcement Learning

Date: Wednesday, Sep 13, 2023, 11:00 - 12:30

Speakers:

• Jacob Beck, University of Oxford
• Risto Vuorio, University of Oxford

Website:

https://sites.google.com/view/meta-rl-tutorial-2023/home

Abstract:

A major drawback of deep reinforcement learning (RL) is its poor data efficiency. In this tutorial, we present meta-RL as an approach to create sample-efficient and general-purpose RL algorithms, via learning the RL algorithm itself. Meta-RL aims to learn a policy that is capable of adapting to any new task from a distribution over tasks with only limited data. We present the meta-RL problem statement, along with an overview of methods, applications, and open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.

Bio:

Jacob Beck is a DPhil (PhD) candidate at the University of Oxford supervised by Shimon Whiteson, funded by the Oxford-Google DeepMind Doctoral Scholarship, and studying meta-reinforcement learning (RL). His work focuses on representations in few-shot meta-RL, with a focus on hypernetworks. He has written a survey paper on meta-RL and was interviewed on the TalkRL podcast, along with his co-author, Risto Vuorio, to explain the field to a broader audience. Previously he completed a pre-doc at Microsoft Research on long-term memory in RL, worked on autonomous vehicles, and completed a BS and MS at Brown University, where he researched RL and was teaching assistant for the graduate-level deep learning course.

Risto Vuorio is a DPhil (PhD) candidate at the University of Oxford supervised by Shimon Whiteson, funded by EPSRC Doctoral Training Partnership Scholarship and Department of Computer Science Scholarship. His research is focused on reinforcement learning with special interests in meta-RL and learning from demonstrations. His work on meta-RL has been recognized with a spotlight at NeurIPS. Together with Jacob Beck, Risto appeared on TalkRL podcast discussing the newest developments in meta-RL. Before Oxford, Risto worked as a research engineer in Satinder Singh’s lab at the University of Michigan and before that at SK T-Brain in Seoul.