StyleBabel: Artistic Style Tagging and Captioning

Ruta, Dan, Gilbert, Andrew, Aggarwal, Pranav, Marri, Naveen, Kale, Ajinkya, Briggs, Jo, Speed, Chris, Jin, Halin, Faieta, Baldo, Filipkowski, Alex, Lin, Zhe and Collomosse, John (2022) StyleBabel: Artistic Style Tagging and Captioning. In: Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VIII. Lecture Notes in Computer Science, 13668 . Springer, Cham, Switzerland, pp. 219-236. ISBN 9783031200731, 9783031200748

[img]
Preview
Text
Dan_Ruta_StyleBabel_ECCV_3_.pdf - Accepted Version

Download (6MB) | Preview
Official URL: https://doi.org/10.1007/978-3-031-20074-8_13

Abstract

We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools. StyleBabel was collected via an iterative method, inspired by ‘Grounded Theory’: a qualitative approach that enables annotation while co-evolving a shared language for fine-grained artistic style attribute description. We demonstrate several downstream tasks for StyleBabel, adapting the recent ALADIN architecture for fine-grained style similarity, to train cross-modal embeddings for: 1) free-form tag generation; 2) natural language description of artistic style; 3) fine-grained text search of style. To do so, we extend ALADIN with recent advances in Visual Transformer (ViT) and cross-modal representation learning, achieving a state of the art accuracy in fine-grained style retrieval.

Item Type: Book Section
Additional Information: ECCV 2022: European Conference on Computer Vision (ECCV); Tel Aviv, Israel; 23-27 Oct 2022
Uncontrolled Keywords: Datasets and evaluation, Image and video retrieval, Vision + language, Vision applications and systems
Subjects: G400 Computer Science
G600 Software Engineering
W200 Design studies
Department: Faculties > Arts, Design and Social Sciences > Design
Related URLs:
Depositing User: John Coen
Date Deposited: 09 Aug 2022 12:00
Last Modified: 12 Nov 2023 03:30
URI: https://nrl.northumbria.ac.uk/id/eprint/49791

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics