Analysis Cloud - Running Sensor Data Analysis Programs on a Cloud Computing Infrastructure

Jan Sipke van der Veen, Bram van der Waaij, Matthijs Vonder, Marc de Jonge, Elena Lazovik, Robert Meijer

2013

Abstract

Sensors have been used for many years to gather information about their environment. The number of sensors connected to the internet is increasing, which has led to a growing demand of data transport and storage capacity. In addition, evermore emphasis is put on processing the data to detect anomalous situations and to identify trends. This paper presents a sensor data analysis platform that executes statistical analysis programs on a cloud computing infrastructure. Compared to existing batch and stream processing platforms, it adds the notion of simulated time, i.e. time that differs from the actual, current time. Moreover, it adds the ability to dynamically schedule the analysis programs based on a single timestamp, recurring schedule, or on the sensor data itself.

References

  1. Akka Website (2012). Akka toolkit for event-driven applications on the jvm. http://akka.io.
  2. Blackwell, W. (2005). A neural-network technique for the retrieval of atmospheric temperature and moisture profiles from high spectral resolution sounding data. IEEE Transactions on Geoscience and Remote Sensing.
  3. Dean, J. and Ghemawat, S. (2004). Mapreduce: Simplified data processing on large clusters. Symposium on Operating Systems Design and Implementation.
  4. Disco Website (2012). Disco distributed computing framework. http://discoproject.org.
  5. Esper Website (2012). Esper complex event processing. http://esper.codehaus.org.
  6. Fielding, R. T. (2000). Architectural styles and the design of network-based software architectures. http:// www.ics.uci.edu/~fielding/pubs/dissertation/fielding dissertation.pdf.
  7. Ghemawat, S., Gobioff, H., and Leung, S.-T. (2003). The google file system. ACM Symposium on Operating Systems Principles.
  8. Hadoop Website (2012). hadoop.apache.org.
  9. Hunt, P., Konar, M., Junqueira, F. P., and Reed, B. (2010). Zookeeper: Wait-free coordination for internet-scale systems. USENIX Annual Technical Conference.
  10. IBM Corporation (2001). Autonomic computing: Ibms perspective on the state of information technology. http://www.research.ibm.com/autonomic/manifesto/ autonomic computing.pdf.
  11. Munish, K. G. (2012). Akka Essentials. Packt Publishing.
  12. Neumeyer, L., Robbins, B., Nair, A., and Kesari, A. (2010). S4: Distributed stream computing platform. IEEE International Conference on Data Mining Workshops.
  13. Pyayt, A., Mokhov, I., Lang, B., Krzhizhanovskaya, V., and Meijer, R. (2011). Machine learning methods for environmental monitoring and flood protection. International Conference on Artificial Intelligence and Neural Networks.
  14. Rao, J., Bu, X., Xu, C.-Z., and Wang, K. (2011). A distributed self-learning approach for elastic provisioning of virtualized cloud resources. IEEE International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems.
  15. S4 Website (2012). S4 distributed stream computing platform. http://incubator.apache.org/s4.
  16. Sheng, B., Li, Q., and Mao, W. (2006). Data storage placement in sensor networks. ACM International Symposium On Mobile Ad Hoc Networking and Computing.
  17. Spark Website (2012). Spark cluster computing framework. http://www.spark-project.org.
  18. Storm Website (2012). Storm distributed realtime computation system. http://storm-project.net.
  19. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., and Stoica, I. (2010). Spark: Cluster computing with working sets. HotCloud.
  20. ZooKeeper Website (2012). zookeeper.apache.org.
Download


Paper Citation


in Harvard Style

van der Veen J., van der Waaij B., Vonder M., de Jonge M., Lazovik E. and Meijer R. (2013). Analysis Cloud - Running Sensor Data Analysis Programs on a Cloud Computing Infrastructure . In Proceedings of the 3rd International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-8565-52-5, pages 358-365. DOI: 10.5220/0004371503580365


in Bibtex Style

@conference{closer13,
author={Jan Sipke van der Veen and Bram van der Waaij and Matthijs Vonder and Marc de Jonge and Elena Lazovik and Robert Meijer},
title={Analysis Cloud - Running Sensor Data Analysis Programs on a Cloud Computing Infrastructure},
booktitle={Proceedings of the 3rd International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2013},
pages={358-365},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004371503580365},
isbn={978-989-8565-52-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Analysis Cloud - Running Sensor Data Analysis Programs on a Cloud Computing Infrastructure
SN - 978-989-8565-52-5
AU - van der Veen J.
AU - van der Waaij B.
AU - Vonder M.
AU - de Jonge M.
AU - Lazovik E.
AU - Meijer R.
PY - 2013
SP - 358
EP - 365
DO - 10.5220/0004371503580365