# Information Theoretic Deductions Using Machine Learning with an Application in Sociology

### Arunselvan Ramaswamy, Yunpeng Ma, Stefan Alfredsson, Fran Collyer, Anna Brunström

#### 2024

#### Abstract

Conditional entropy is an important concept that naturally arises in fields such as finance, sociology, and intelligent decision making when solving problems involving statistical inferences. Formally speaking, given two random variables X and Y, one is interested in the amount and direction of information flow between X and Y. It helps to draw conclusions about Y while only observing X. Conditional entropy H(Y|X) quantifies the amount of information flow from X to Y. In practice, calculating H(Y|X) exactly is infeasible. Current estimation methods are complex and suffer from estimation bias issues. In this paper, we present a simple Machine Learning based estimation method. Our method can be used to estimate H(Y|X) for discrete X and bi-valued Y. Given X and Y observations, we first construct a natural binary classification training dataset. We then train a supervised learning algorithm on this dataset, and use its prediction accuracy to estimate H(Y|X). We also present a simple condition on the prediction accuracy to determine if there is information flow from X to Y. We support our ideas using formal arguments and through an experiment involving a gender-bias study using a part of the employee database of Karlstad University, Sweden.

Download#### Paper Citation

#### in Harvard Style

Ramaswamy A., Ma Y., Alfredsson S., Collyer F. and Brunström A. (2024). **Information Theoretic Deductions Using Machine Learning with an Application in Sociology**. In *Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM*; ISBN 978-989-758-684-2, SciTePress, pages 320-328. DOI: 10.5220/0012371600003654

#### in Bibtex Style

@conference{icpram24,

author={Arunselvan Ramaswamy and Yunpeng Ma and Stefan Alfredsson and Fran Collyer and Anna Brunström},

title={Information Theoretic Deductions Using Machine Learning with an Application in Sociology},

booktitle={Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},

year={2024},

pages={320-328},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0012371600003654},

isbn={978-989-758-684-2},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM

TI - Information Theoretic Deductions Using Machine Learning with an Application in Sociology

SN - 978-989-758-684-2

AU - Ramaswamy A.

AU - Ma Y.

AU - Alfredsson S.

AU - Collyer F.

AU - Brunström A.

PY - 2024

SP - 320

EP - 328

DO - 10.5220/0012371600003654

PB - SciTePress