Pre-Training and Fine-Tuning Attention Based Encoder Decoder Improves Sea Surface Height Multi-Variate Inpainting

Théo Archambault, Théo Archambault, Arthur Filoche, Arthur Filoche, Anastase Charantonis, Anastase Charantonis, Dominique Béréziat

2024

Abstract

The ocean is observed through satellites measuring physical data of various natures. Among them, Sea Surface Height (SSH) and Sea Surface Temperature (SST) are physically linked data involving different remote sensing technologies and therefore different image inverse problems. In this work, we propose to use an Attention-based Encoder-Decoder to perform the inpainting of the SSH, using the SST as contextual information. We propose to pre-train this neural network on a realistic twin experiment of the observing system and to fine-tune it in an unsupervised manner on real-world observations. We show the interest of this strategy by comparing it to existing methods. Our training methodology achieves state-of-the-art performances, and we report a decrease of 25% in error compared to the most widely used interpolations product.

Paper Citation

in Harvard Style

Archambault T., Filoche A., Charantonis A. and Béréziat D. (2024). Pre-Training and Fine-Tuning Attention Based Encoder Decoder Improves Sea Surface Height Multi-Variate Inpainting. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 100-109. DOI: 10.5220/0012357400003660

in Bibtex Style

@conference{visapp24,
author={Théo Archambault and Arthur Filoche and Anastase Charantonis and Dominique Béréziat},
title={Pre-Training and Fine-Tuning Attention Based Encoder Decoder Improves Sea Surface Height Multi-Variate Inpainting},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2024},
pages={100-109},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012357400003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - Pre-Training and Fine-Tuning Attention Based Encoder Decoder Improves Sea Surface Height Multi-Variate Inpainting
SN - 978-989-758-679-8
AU - Archambault T.
AU - Filoche A.
AU - Charantonis A.
AU - Béréziat D.
PY - 2024
SP - 100
EP - 109
DO - 10.5220/0012357400003660
PB - SciTePress