SELF-ORGANIZING SYNCHRONICITY AND DESYNCHRONICITY USING REINFORCEMENT LEARNING

Mihail Mihaylov, Yann-Aël Le Borgne, Ann Nowé, Karl Tuyls

Abstract

We present a self-organizing reinforcement learning (RL) approach for coordinating the wake-up cycles of nodes in a wireless sensor network in a decentralized manner. To the best of our knowledge we are the first to demonstrate how global synchronicity and desynchronicity can emerge through local interactions alone without the need of central mediator or any form of explicit coordination. We apply this RL approach to wireless sensor nodes arranged in different topologies and study how agents, starting with a random policy, are able to self-adapt their behavior based only on their interaction with neighboring nodes. Each agent independently learns to which nodes it should synchronize to improve message throughput and at the same with whom to desynchronize in order to reduce communication interference. The obtained results show how simple and computationally bounded sensor nodes are able to coordinate their wake-up cycles in a distributed way in order to improve the global system performance through (de)synchronicity.

References

  1. Cohen, R. and Kapchits, B. (2009). An optimal wakeup scheduling algorithm for minimizing energy consumption while limiting maximum delay in a mesh sensor network. IEEE/ACM Trans. Netw., 17(2):570- 581.
  2. Degesys, J., Rose, I., Patel, A., and Nagpal, R. (2007). Desync: self-organizing desynchronization and tdma on wireless sensor networks. In Proceedings of the 6th IPSN, pages 11-20, New York, NY, USA. ACM.
  3. Ilyas, M. and Mahgoub, I. (2005). Handbook of sensor networks: compact wireless and wired sensing systems. CRC.
  4. Knoester, D. B. and McKinley, P. K. (2009). Evolving virtual fireflies. In Proceedings of the 10th ECAL, Budapest, Hungary.
  5. Langendoen, K. (2008). Medium access control in wireless sensor networks. Medium access control in wireless networks, 2:535-560.
  6. Leng, J. (2008). Reinforcement learning and convergence analysis with applications to agent-based systems. PhD thesis, University of South Australia.
  7. Liang, S., Tang, Y., and Zhu, Q. (2007). Passive wake-up scheme for wireless sensor networks. In Proceedings of the 2nd ICICIC, page 507, Washington, DC, USA. IEEE Computer Society.
  8. Liu, Z. and Elhanany, I. (2006). Rl-mac: a reinforcement learning based mac protocol for wireless sensor networks. Int. J. Sen. Netw., 1(3/4):117-124.
  9. Lucarelli, D. and Wang, I.-J. (2004). Decentralized synchronization protocols with nearest neighbor communication. In Proceedings of the 2nd SenSys, pages 62- 68, New York, NY, USA. ACM.
  10. Mirollo, R. E. and Strogatz, S. H. (1990). Synchronization of pulse-coupled biological oscillators. SIAM J. Appl. Math., 50(6):1645-1662.
  11. Paruchuri, V., Basavaraju, S., Durresi, A., Kannan, R., and Iyengar, S. S. (2004). Random asynchronous wakeup protocol for sensor networks. In Proceedings of the 1st BROADNETS, pages 710-717, Washington, DC, USA. IEEE Computer Society.
  12. Patel, A., Degesys, J., and Nagpal, R. (2007). Desynchronization: The theory of self-organizing algorithms for round-robin scheduling. In Proceedings of the 1st SASO, pages 87-96, Washington, DC, USA. IEEE Computer Society.
  13. Schurgers, C. (2007). Wireless Sensor Networks and Applications, chapter Wakeup Strategies in Wireless Sensor Networks, page 26. Springer.
  14. Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: an Introduction. MIT Press.
  15. Watkins, C. (1989). Learning from delayed rewards. PhD thesis, University of Cambridge, England.
  16. Werner-Allen, G., Tewari, G., Patel, A., Welsh, M., and Nagpal, R. (2005). Firefly-inspired sensor network synchronicity with realistic radio effects. In Proceedings of the 3rd SenSys, pages 142-153, New York, NY, USA. ACM.
  17. Wolpert, D. H. and Tumer, K. (2008). An introduction to collective intelligence. Technical Report NASAARC-IC-99-63, NASA Ames Research Center.
  18. Ye, W., Heidemann, J., and Estrin, D. (2004). Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Trans. Netw., 12(3):493-506.
  19. Zheng, R., Hou, J. C., and Sha, L. (2003). Asynchronous wakeup for ad hoc networks. In Proceedings of the 4th MobiHoc, pages 35-45, New York, NY, USA. ACM.
Download


Paper Citation


in Harvard Style

Mihaylov M., Le Borgne Y., Nowé A. and Tuyls K. (2011). SELF-ORGANIZING SYNCHRONICITY AND DESYNCHRONICITY USING REINFORCEMENT LEARNING . In Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-8425-41-6, pages 94-103. DOI: 10.5220/0003162600940103


in Bibtex Style

@conference{icaart11,
author={Mihail Mihaylov and Yann-Aël Le Borgne and Ann Nowé and Karl Tuyls},
title={SELF-ORGANIZING SYNCHRONICITY AND DESYNCHRONICITY USING REINFORCEMENT LEARNING},
booktitle={Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2011},
pages={94-103},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003162600940103},
isbn={978-989-8425-41-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - SELF-ORGANIZING SYNCHRONICITY AND DESYNCHRONICITY USING REINFORCEMENT LEARNING
SN - 978-989-8425-41-6
AU - Mihaylov M.
AU - Le Borgne Y.
AU - Nowé A.
AU - Tuyls K.
PY - 2011
SP - 94
EP - 103
DO - 10.5220/0003162600940103