Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning

Nozawa, Naoiki, Shum, Hubert, Ho, Edmond and Morishima, Shigeo (2020) Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning. In: Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 1: GRAPP. SciTePress, pp. 179-190. ISBN 9789897584022

[img]
Preview
Text
GRAPP_2020_46.pdf - Published Version
Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0.

Download (90MB) | Preview
[img] Text
GRAPP 2020 Compressed.pdf - Accepted Version
Restricted to Repository staff only until 20 December 2020.

Download (1MB) | Request a copy
Official URL: https://doi.org/10.5220/0009157001790190

Abstract

Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketchimage. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deepneural network that takes a 2D sketch and generates a set of multi-view depth and mask images, which forma more effective representation comparing to 3D meshes, and can be effectively fused to generate a 3D carshape. Since global models like deep learning have limited capacity to reconstruct fine-detail features, wepropose a local lazy learning approach that constructs a small subspace based on a few relevant car samples inthe database. Due to the small size of such a subspace, fine details can be represented effectively with a smallnumber of parameters. With a low-cost optimization process, a high-quality car shape with detailed featuresis created. Experimental results show that the system performs consistently to create highly realistic cars ofsubstantially different shape and topology.

Item Type: Book Section
Uncontrolled Keywords: Deep Learning, Lazy Learning, 3D Reconstruction, Sketch-based Interface, Car
Subjects: G400 Computer Science
G500 Information Systems
H700 Production and Manufacturing Engineering
Department: Faculties > Engineering and Environment > Computer and Information Sciences
Depositing User: Elena Carlaw
Date Deposited: 06 Jan 2020 09:58
Last Modified: 02 Jun 2020 12:15
URI: http://nrl.northumbria.ac.uk/id/eprint/41820

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics