Rapid Learning of Complex Control Tasks based on Online Optimization Meta-Learning

• Zhixiong Xu,
• Ailing Li,
• Zongming Guo,
• Zhiruo Zhao,
• Qianrui Qi,
• Xiliang Chen,

Vol. 19, No. 7, July 31, 2025
10.3837/tiis.2025.07.002, Download Paper (Free):

• Control tasks
• Meta-Learning
• online hyper -parameter adaptation
• Deep Reinforcement Learning

Abstract

Recent deep reinforcement learning (DRL) methods demonstrate outstanding performance in a variety of fields, however, most algorithms suffer from a drawback of inefficient learning that limits the application of DRL. Since meta-learning tackles this problem by leveraging prior experience to help DRL models quickly adapt to unseen tasks, this paper focuses on addressing the problem of time-consuming hyper-parameter tuning for training stability in optimization-based meta-learning. A novel and versatile online learning rate adaptation (OLA) method is presented based on one of the most prominent optimization-based meta-learning named model-agnostic meta-learning, in order to eliminate the need to tune the learning rate and meta-learning rate. Specifically, an online hyper-parameter adaptation mechanism is incorporated into optimization-based meta-learning, which not only learns a learning rate and direction for each layer based on stochastic gradient descent method, but also performs online optimization towards the meta-learning rate according to hyper-gradient descent method. In addition, the sensitivity of the proposed approach with respect to the depth of network model and hyper-parameter choice are also discussed respectively. The experimental results with a variety of challenging control tasks demonstrate that the proposed algorithm yields models with highly competitive performance and generalization for reinforcement learning.

Statistics

Cite this article