3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning

Nozawa, Naoiki, Shum, Hubert, Feng, Qi, Ho, Edmond and Morishima, Shigeo (2020) 3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning. The Visual Computer. ISSN 0178-2789 (In Press)

[img] Text
grapp_extendded_naoki_1_.pdf - Accepted Version
Restricted to Repository staff only until 3 September 2021.

Download (24MB) | Request a copy

Abstract

3D car models are heavily used in computer games, visual effects, and even automotive designs. As a result, producing such models with minimal labour costs is increasingly more important. To tackle the challenge, we propose a novel system to reconstruct a 3D car using a single sketch image. The system learns from a synthetic database of 3D car models and their corresponding 2D contour sketches and segmentation masks, allowing effective training with minimal data collection cost. The core of the system is a machine learning pipeline that combines the use of a Generative Adversarial Network (GAN) and lazy learning. GAN, being a deep learning method, is capable of modelling complicated data distributions, enabling the effective modelling of a large variety of cars. Its major weakness is that as a global method, modelling the fine details in the local region is challenging. Lazy learning works well to preserve local features by generating a local subspace with relevant data samples. We demonstrate that the combined use of GAN and lazy learning produces is able to produce high-quality results, in which different types of cars with complicated local features can be generated effectively with a single sketch. Our method outperforms existing ones using other machine learning structures such as the variational autoencoder.

Item Type: Article
Uncontrolled Keywords: Generative Adversarial Network, Lazy Learning, 3D Reconstruction, Sketch-based Interface, Car, Contour Sketch
Subjects: G400 Computer Science
G500 Information Systems
G700 Artificial Intelligence
G900 Others in Mathematical and Computing Sciences
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Rachel Branson
Date Deposited: 14 Oct 2020 09:41
Last Modified: 14 Oct 2020 09:45
URI: http://nrl.northumbria.ac.uk/id/eprint/44507

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics