Authors:
Leandro Stival
1
;
Ricardo Torres
2
;
3
and
Helio Pedrini
1
Affiliations:
1
Institute of Computing, University of Campinas, Av. Albert Einstein 1251, Campinas, SP, 13083-852, Brazil
;
2
Wageningen Data Competence Center, Wageningen University and Research, Wageningen, The Netherlands
;
3
Norwegian University of Science and Technology, Larsgårdsvegen 2, 6009 Alesund, Norway
Keyword(s):
Video Colorization, Deep Learning, Cosine Similarity, Attention Mechanism.
Abstract:
Video colorization is a challenging task, demanding deep learning models to employ diverse abstractions for a comprehensive grasp of the task, ultimately yielding high-quality results. Currently, in example-based colorization approaches, the use of attention processes and convolutional layers has proven to be the most effective method to produce good results. Following this way, in this paper we propose Cosine Attention Video Colorization (CAVC), which is an approach that uses a single attention head with shared weights to produce a refinement of the monochromatic frame, as well as the cosine similarity between this sample and the other channels present in the image. This entire process acts as a pre-processing of the data from our autoencoder, which performs a feature fusion with the latent space extracted from the referent frame, as well as with its histogram. This architecture was trained on the DAVIS, UVO and LDV datasets and achieved superior results compared to state-of-the-art
models in terms of FID metric in all the datasets.
(More)