Explicit Dynamic Coordination Reinforcement Learning Based on Utility

• Huaiwei Si,
• Guozhen Tan,
• Yifu Yuan,
• Yanfei peng,
• Jianping Li,

Vol. 16, No. 3, March 31, 2022
10.3837/tiis.2022.03.003, Download Paper (Free):

• Reinforcement Learning
• Multi-agent system
• Explicit Coordination Learning
• Utility Dependence
• Intelligent Vehicle

Abstract

Multi-agent systems often need to achieve the goal of learning more effectively for a task through coordination. Although the introduction of deep learning has addressed the state space problems, multi-agent learning remains infeasible because of the joint action spaces. Large-scale joint action spaces can be sparse according to implicit or explicit coordination structure, which can ensure reasonable coordination action through the coordination structure. In general, the multi-agent system is dynamic, which makes the relations among agents and the coordination structure are dynamic. Therefore, the explicit coordination structure can better represent the coordinative relationship among agents and achieve better coordination between agents. Inspired by the maximization of social group utility, we dynamically construct a factor graph as an explicit coordination structure to express the coordinative relationship according to the utility among agents and estimate the joint action values based on the local utility transfer among factor graphs. We present the application of such techniques in the scenario of multiple intelligent vehicle systems, where state space and action space are a problem and have too many interactions among agents. The results on the multiple intelligent vehicle systems demonstrate the efficiency and effectiveness of our proposed methods.

Statistics

Cite this article