loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: Changqing Fu and Laurent Cohen

Affiliation: CEREMADE, University Paris Dauphine, PSL Research University, UMR CNRS 7534, Paris, 75016, France

Keyword(s): Generative AI, Deep Learning for Visual Understanding, Machine Learning Technologies for Vision.

Abstract: We introduce Conic Linear Unit (CoLU), a natural generalization of commonly used activation functions in neural networks. The common pointwise ReLU activation is a projection onto the positive cone and is permutation symmetric. We propose a nonlinearity that goes beyond this symmetry: CoLU is a skew projection onto a hypercone towards the cone’s axis. Due to the nature of this projection, CoLU enforces symmetry in a neural network with width C from the finite-order permutation group S(C) to the infinite-order rotation/reflection group O(C− 1), thus producing deep features that are motivated by the HSV color representation. Recent results on merging independent neural networks via permutation modulus can be relaxed and generalized to soft alignment modulo an optimal transport plan (Singh and Jaggi, 2020), which is useful in aligning models of different widths. CoLU aims to further alleviate the apparent deficiency of soft alignment. Our simulation indicates that CoLU outperforms exist ing generative models including Autoencoder and Latent Diffusion Model on small or large-scale image datasets. Additionally, CoLU does not increase the number of parameters and requires negligible additional computation overhead. The CoLU concept is quite general and can be plugged into various neural network architectures. Ablation studies on extensions to soft projections, general L p cones, and the non-convex double-cone cases are briefly discussed. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 44.211.117.197

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Fu, C. and Cohen, L. (2024). Conic Linear Units: Improved Model Fusion and Rotational-Symmetric Generative Model. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8; ISSN 2184-4321, SciTePress, pages 686-693. DOI: 10.5220/0012406500003660

@conference{visapp24,
author={Changqing Fu. and Laurent Cohen.},
title={Conic Linear Units: Improved Model Fusion and Rotational-Symmetric Generative Model},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={686-693},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012406500003660},
isbn={978-989-758-679-8},
issn={2184-4321},
}

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Conic Linear Units: Improved Model Fusion and Rotational-Symmetric Generative Model
SN - 978-989-758-679-8
IS - 2184-4321
AU - Fu, C.
AU - Cohen, L.
PY - 2024
SP - 686
EP - 693
DO - 10.5220/0012406500003660
PB - SciTePress