loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: Bruno Casella 1 ; 2 ; Alessio Barbaro Chisari 3 ; 4 ; Sebastiano Battiato 4 and Mario Valerio Giuffrida 5

Affiliations: 1 Department of Computer Science, University of Torino, Torino, Italy ; 2 Department of Economics and Business, University of Catania, Catania, Italy ; 3 Department of Civil Engineering and Architecture, University of Catania, Catania, Italy ; 4 Department of Mathematics and Computer Science, University of Catania, Catania, Italy ; 5 School of Computing, Edinburgh Napier University, Edinburgh, U.K.

Keyword(s): Parameter Aggregation, Transfer Learning, Selective Forgetting.

Abstract: It has been demonstrated that deep neural networks outperform traditional machine learning. However, deep networks lack generalisability, that is, they will not perform as good as in a new (testing) set drawn from a different distribution due to the domain shift. In order to tackle this known issue, several transfer learning approaches have been proposed, where the knowledge of a trained model is transferred into another to improve performance with different data. However, most of these approaches require additional training steps, or they suffer from catastrophic forgetting that occurs when a trained model has overwritten previously learnt knowledge. We address both problems with a novel transfer learning approach that uses network aggregation. We train dataset-specific networks together with an aggregation network in a unified framework. The loss function includes two main components: a task-specific loss (such as cross-entropy) and an aggregation loss. The proposed aggregation los s allows our model to learn how trained deep network parameters can be aggregated with an aggregation operator. We demonstrate that the proposed approach learns model aggregation at test time without any further training step, reducing the burden of transfer learning to a simple arithmetical operation. The proposed approach achieves comparable performance w.r.t. the baseline. Besides, if the aggregation operator has an inverse, we will show that our model also inherently allows for selective forgetting, i.e., the aggregated model can forget one of the datasets it was trained on, retaining information on the others. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 44.192.20.240

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Casella, B.; Chisari, A.; Battiato, S. and Giuffrida, M. (2022). Transfer Learning via Test-time Neural Networks Aggregation. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5; ISSN 2184-4321, SciTePress, pages 642-649. DOI: 10.5220/0010907900003124

@conference{visapp22,
author={Bruno Casella. and Alessio Barbaro Chisari. and Sebastiano Battiato. and Mario Valerio Giuffrida.},
title={Transfer Learning via Test-time Neural Networks Aggregation},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={642-649},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010907900003124},
isbn={978-989-758-555-5},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Transfer Learning via Test-time Neural Networks Aggregation
SN - 978-989-758-555-5
IS - 2184-4321
AU - Casella, B.
AU - Chisari, A.
AU - Battiato, S.
AU - Giuffrida, M.
PY - 2022
SP - 642
EP - 649
DO - 10.5220/0010907900003124
PB - SciTePress