Authors:
Ana Rita Bamansá Siles Machado
1
;
Heitor Cardoso
2
;
Plinio Moreno
2
and
Alexandre Bernardino
2
Affiliations:
1
Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, Portugal
;
2
Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, Torre Norte Piso 7, 1049-001 Lisboa, Portugal
Keyword(s):
mHealth, Notifications, Machine Learning, Personalization, Reinforcement Learning, Receptivity.
Abstract:
This paper aims to develop an intelligent notification system to help sustain user engagement in mHealth applications, specifically those that support self-management. We rely on Reinforcement Learning (RL), an approach where agent learns by exploration the most opportune time to perform a questionnaire, throughout their day, only from easily obtainable non-sensitive data and usage history. This history allows the agent to remember how the user reacts or has reacted in the past to its actions. We consider several options on algorithm, state representation and reward function under the RL umbrella (Upper Confidence Bound, Tabular Q-learning and Deep Q-learning). In addition, a simulator was developed to mimic the behavior of a typical user and utilized to test all possible combinations with users experiencing distinct lifestyles. We obtain promising promising results, which still requiring further testing to be fully validated. We demonstrate that an efficient and well-balanced notifi
cation system can be built with simple formulations of an RL problem and algorithm. Furthermore, our approach does not require to have access to sensitive user data. This approach diminishes privacy issues that might concern the user and limits sensor and hardware concerns, such as lapses in collected data or battery drainage.
(More)