Tensor optimization with group lasso for multi-agent predictive state representation

Ma, Biyang, Tang, Jing, Chen, Bilian, Pan, Yinghui and Zeng, Yifeng (2021) Tensor optimization with group lasso for multi-agent predictive state representation. Knowledge-Based Systems, 221. p. 106893. ISSN 0950-7051

template.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0.

Download (1MB) | Preview
Official URL: https://doi.org/10.1016/j.knosys.2021.106893


Predictive state representation (PSR) is a compact model of dynamic systems that represents state as a vector of predictions about future observable events. It is an alternative to a partially observable Markov decision process (POMDP) model in dealing with a sequential decision-making problem under uncertainty. Most of the existing PSR research focuses on the model learning in a single-agent setting. In this paper, we investigate a multi-agent PSR model upon available agents interaction data. It turns out to be rather difficult to learn a multi-agent PSR model especially with limited samples and increasing number of agents. We resort to a tensor technique to better represent dynamic system characteristics and address the challenging task of learning multi-agent PSR problems based on tensor optimization. We first focus on a two-agent scenario and use a third order tensor (system dynamics tensor) to capture the system interaction data. Then, the PSR model discovery can be formulated as a tensor optimization problem with group lasso, and an alternating direction method of multipliers is called for solving the embedded subproblems. Hence, the prediction parameters and state vectors can be directly learned from the optimization solutions, and the transition parameters can be derived via a linear regression. Subsequently, we generalize the tensor learning approach in a multi(N >2)-agent PSR model, and analyze the computational complexity of the learning algorithms. Experimental results show that the tensor optimization approaches have provided promising performances on learning a multi-agent PSR model over multiple problem domains.

Item Type: Article
Additional Information: Funding information: Dr. Bilian Chen and Dr. Yinghui Pan were supported in part by the National Natural Science Foundation of China (Grants No. 61772442, 61806089 and 61836005). Professor Yifeng Zeng and Dr. Biyang Ma thanks the support of the EPSRC New Investigator Award in 2019.
Uncontrolled Keywords: predictive state representations, tensor optimization, alternating direction method of multipliers, group lasso
Subjects: G400 Computer Science
Department: Faculties > Business and Law > Newcastle Business School
Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: John Coen
Date Deposited: 22 Mar 2021 08:43
Last Modified: 17 Mar 2022 03:30
URI: http://nrl.northumbria.ac.uk/id/eprint/45747

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics