Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration

Jiang, Feibo, Dong, Li, Wang, Kezhi, Yang, Kun and Pan, Cunhua (2021) Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration. IEEE Internet of Things Journal. pp. 1-14. ISSN 2372-2541 (In Press)

[img]
Preview
Text
Distributed_Resource_Scheduling_for_Large-Scale_MEC_Systems_A_Multi-Agent_Ensemble_Deep_Reinforcement_Learning_with_Imitation_Acceleration.pdf - Accepted Version

Download (1MB) | Preview
Official URL: https://doi.org/10.1109/JIOT.2021.3113872

Abstract

In large-scale mobile edge computing (MEC) systems, the task latency and energy consumption are important for massive resource-consuming and delay-sensitive Internet of things devices (IoTDs). Against this background, we propose a distributed intelligent resource scheduling (DIRS) framework to minimize the sum of task latency and energy consumption for all IoTDs, which can be formulated as a mixed integer nonlinear programming. The DIRS framework includes centralized training relying on the global information and distributed decision making by each agent deployed in each MEC server. Specifically, we first introduce a novel multi-agent ensemble-assisted distributed deep reinforcement learning (DRL) architecture, which can simplify the overall neural network structure of each agent by partitioning the state space and also improve the performance of a single agent by combining decisions of all the agents. Secondly, we apply action refinement to enhance the exploration ability of the proposed DIRS framework, where the near-optimal state-action pairs are obtained by a novel Levy flight search. Finally, an imitation acceleration scheme is presented to pre-train all the agents, which can significantly accelerate the learning process of the proposed framework through learning the professional experience from a small amount of demonstration data. The simulation results in three typical scenarios demonstrate that the proposed DIRS framework is efficient and outperforms the existing benchmark schemes.

Item Type: Article
Additional Information: Funding information: This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant no. 41604117, 41904127, 61620106011 and U1705263; in part by the Hunan Provincial Natural Science Foundation of China under Grant no. 2020JJ4428, 2020JJ5105, 2021JJ30455; in part by the Hunan Provincial Science Technology Project Foundation under Grant 2018TP1018 and 2018RS3065.
Uncontrolled Keywords: Multi-agent reinforcement learning, Distributed deep reinforcement learning, Imitation learning, Resource scheduling, Levy flight.
Subjects: G400 Computer Science
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: John Coen
Date Deposited: 13 Dec 2021 11:39
Last Modified: 13 Dec 2021 11:45
URI: http://nrl.northumbria.ac.uk/id/eprint/47965

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics