Video foreground segmentation with deep learning

Sakkos, Dimitris (2020) Video foreground segmentation with deep learning. Doctoral thesis, Nothumbria University.

Text (Doctoral Thesis)
sakkos.dimitrios_phd.pdf - Submitted Version

Download (29MB) | Preview


This thesis tackles the problem of foreground segmentation in videos, even under extremely challenging conditions. This task comes with a plethora of hurdles, as the model needs to distinguish the difference between moving objects and irrelevant background motion which can be caused by the weather, illumination, camera movement etc. As foreground segmentation is often the first step of various highly important applications (video surveillance for security, patient/infant monitoring etc.), it is crucial to develop a model capable of producing excellent results in all kinds of conditions.

In order to tackle this problem, we follow the recent trend in other computer vision areas and harness the power of deep learning. We design architectures of convolutional neural networks specifically targeted to counter the aforementioned challenges. We first propose a 3D CNN that models the spatial and temporal information of the scene simultaneously. The network is deep enough to successfully cover more than 50 different scenes of various conditions with no need for any fine-tuning. These conditions include illumination (day or night), weather (sunny, rainy or snowing), background movements (trees moving from the wind, fountains etc) and others. Next, we propose a data augmentation method specifically targeted to illumination changes. We show that artificially augmenting the data set with this method significantly improves the segmentation results, even when tested under sudden illumination changes. We also present a post-processing method that exploits the temporal information of the input video. Finally, we propose a complex deep learning model which learns the illumination of the scene and performs foreground segmentation simultaneously.

Item Type: Thesis (Doctoral)
Uncontrolled Keywords: Background Subtraction, Generative Adversarial Networks, Illumination based data augmentation, 3D Convolutional Neural Networks
Subjects: G400 Computer Science
G600 Software Engineering
G700 Artificial Intelligence
Department: Faculties > Engineering and Environment > Computer and Information Sciences
University Services > Graduate School > Doctor of Philosophy
Depositing User: John Coen
Date Deposited: 05 May 2020 09:27
Last Modified: 31 Jul 2021 18:16

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics