An LSTM Based Generative Adversarial Architecture for Robotic Calligraphy Learning System

Chao, Fei, Lin, Gan, Zheng, Ling, Chang, Xiang, Lin, Chih-Min, Yang, Longzhi and Shang, Changjing (2020) An LSTM Based Generative Adversarial Architecture for Robotic Calligraphy Learning System. Sustainability, 12 (21). p. 9092. ISSN 2071-1050

sustainability-12-09092.pdf - Published Version
Available under License Creative Commons Attribution 4.0.

Download (3MB) | Preview
Official URL:


Robotic calligraphy is a very challenging task for the robotic manipulators, which can sustain industrial manufacturing. The active mechanism of writing robots require a large sized training set including sequence information of the writing trajectory. However, manual labelling work on those training data may cause the time wasting for researchers. This paper proposes a machine calligraphy learning system using a Long Short-Term Memory (LSTM) network and a generative adversarial network (GAN), which enables the robots to learn and generate the sequences of Chinese character stroke (i.e., writing trajectory). In order to reduce the size of the training set, a generative adversarial architecture combining an LSTM network and a discrimination network is established for a robotic manipulator to learn the Chinese calligraphy regarding its strokes. In particular, this learning system converts Chinese character stroke image into the trajectory sequences in the absence of the stroke trajectory writing sequence information. Due to its powerful learning ability in handling motion sequences, the LSTM network is used to explore the trajectory point writing sequences. Each generation process of the generative adversarial architecture contains a number of loops of LSTM. In each loop, the robot continues to write by following a new trajectory point, which is generated by LSTM according to the previously written strokes. The written stroke in an image format is taken as input to the next loop of the LSTM network until the complete stroke is finally written. Then, the final output of the LSTM network is evaluated by the discriminative network. In addition, a policy gradient algorithm based on reinforcement learning is employed to aid the robot to find the best policy. The experimental results show that the proposed learning system can effectively produce a variety of high-quality Chinese stroke writing.

Item Type: Article
Uncontrolled Keywords: Robotic calligraphy system; robotic learning; motion planning; Long Short-Term Memory network; adversarial learning
Subjects: G400 Computer Science
G500 Information Systems
G700 Artificial Intelligence
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Elena Carlaw
Date Deposited: 02 Nov 2020 08:59
Last Modified: 31 Jul 2021 13:18

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics