Bernoulli Distribution-Based Maximum Likelihood Estimation for Dynamic Coefficient Optimization in Model-Contrastive Federated Learning

Sichong Liao

2024

Abstract

In the realm of federated learning, the non-identically and independently distributed (non-IID) nature of data presents a formidable challenge, often leading to suboptimal model performance. This study introduces a novel Bernoulli Distribution-Based Maximum Likelihood Estimation for Dynamic Coefficient Optimization method in Model-Contrastive Federated Learning, aiming to address these inherent difficulties. The center of the proposed approach is the dynamic adjustment of loss terms concurring to quantifying deviation between the global model and local model. There may be a lot of variation in the data. In this case, the proposed manner could upgrade the robustness and adaptability of the model itself. Leveraging a Model-Contrastive Federated Learning (MOON) framework, this paper proposed a Dynamic Coefficient Optimized MOON (DCO-MOON) framework. For the supervised loss term and model-contrastive loss term, the proposed approach incorporates a dynamic coefficient adjustment mechanism. The efficacy of this approach is illustrated through the simulations on different datasets, including the Modified National Institute of Standards and Technology (MNIST), Fashion-MNIST, and Canadian Institute for Advanced Research (CIFAR-10). Experimental results show improvements in test accuracy and communication efficiency. It also illustrates that DCO-MOON can superiorly adjust to real-world scenarios, which are confronting data-driven challenges with non-IID and unbalanced datasets.

Paper Citation

in Harvard Style

Liao S. (2024). Bernoulli Distribution-Based Maximum Likelihood Estimation for Dynamic Coefficient Optimization in Model-Contrastive Federated Learning. In Proceedings of the 1st International Conference on Data Science and Engineering - Volume 1: ICDSE; ISBN 978-989-758-690-3, SciTePress, pages 504-511. DOI: 10.5220/0012827600004547

in Bibtex Style

@conference{icdse24,
author={Sichong Liao},
title={Bernoulli Distribution-Based Maximum Likelihood Estimation for Dynamic Coefficient Optimization in Model-Contrastive Federated Learning},
booktitle={Proceedings of the 1st International Conference on Data Science and Engineering - Volume 1: ICDSE},
year={2024},
pages={504-511},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012827600004547},
isbn={978-989-758-690-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Data Science and Engineering - Volume 1: ICDSE
TI - Bernoulli Distribution-Based Maximum Likelihood Estimation for Dynamic Coefficient Optimization in Model-Contrastive Federated Learning
SN - 978-989-758-690-3
AU - Liao S.
PY - 2024
SP - 504
EP - 511
DO - 10.5220/0012827600004547
PB - SciTePress