Integrated Analytics for Application Management using Stream Clustering and Semantics

M. Omair Shafiq

Abstract

Large-scale software applications produce enormous amount of execution data in the form of logs which makes it challenging for managing execution of such applications. There have been several semantically enhanced analytical solutions proposed for enhanced monitoring and management of software applications. In this paper, author proposes a customized semantic model for representing application execution, and a scalable stream clustering based processing solution. The stream clustering based approach acts as key to combine all the other analytical solutions using the proposed customized semantic model for logs. The proposed approach works in an integrated manner that clusters log data that is produced, as a result of events occurring during execution, at a large-scale and in a continuous streaming manner for managing execution of software applications. The proposed solution utilizes semantics for better expressiveness of log events, other related data and analytical approaches, through stream clustering based integrated approach, to process logs that helps in enhancing the process of monitoring and management of software applications. This paper presents the customized semantic logging model for scalable stream clustering, algorithm design and discussion on scalable stream clustering based solution and its integration with other analytical solutions. The paper also presents experimentation, evaluation and demonstrates applicability of the proposed solution.

References

  1. W3C Semantic Web activity, 2001. W3C Recommendations on RDF and OWL. Available at http://www.w3.org/2001/sw
  2. The Object Management Group, 2002. Meta-Object Facility, version 1.4, 2002. Available at http://www.omg.org/technology/documents/formal/mo f.htm
  3. Semantic Web Service Framework, 2005. SWSF version 1.0. SWSF Available from http://www.daml.org/services/swsf/1.0/, 2005.
  4. de Bruijn, J., 2005. D16 WSML specification. WSMO Deliverable available at http://www.wsmo.org/TR/d16/, February 2005.
  5. Mocan, A., Moran, M., Cimpian, E., Zaremba. M., 2006. Filling the gap - extending service oriented architectures with semantics. In IEEE International Conference on e-Business Engineering 2006 (ICEBE 2006), pages 594-601, Oct 2006.
  6. Friedman, N., Geiger, D., Goldszmidt, M., 1997. Bayesian Network Classifiers, Journal of Machine Learning, vol. 29, pages 131-163, November 1997.
  7. Vaarandi, R., 2003. A Data Clustering Algorithm for Mining Patterns From Event Logs, 2003 IEEE Workshop on IP Operations and Management (IPOM 2003), 1-3 October 2003, Kansas City, Missouri, USA.
  8. Hand, D., Mannila, H., Smyth, P., 2001. Principles of Data Mining, The MIT Press, 2001.
  9. Berkhin, P., “Survey of Clustering Data Mining Techniques”, (see http://citeseer.nj.nec.com/berkhin02survey.html), 2002.
  10. Makanju, A., Brooks, S., Nur Zincir-Heywood, A., Milios, E., 2008. LogView: Visualizing Event Log Clusters, Conference on Privacy, Security and Trust (PST 2008), Fredericton, NB, Canada.
  11. Vaarandi, R., 2003. A Data Clustering Algorithm for Mining Patterns From Event Logs, IEEE Workshop on IP Operations and Management (IPOM 2003), 1-3 Oct 2003, Kansas City, Missouri, USA.
  12. Beeferman, D., Berger, A., 2000. Agglomerative clustering of a search engine query log, 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 407-416, 20-23 August 2000, Boston, MA, USA.
  13. Paolucci, M., Srinivasan, N., Sycara, K., Nishimura, T., 2003. Towards a Semantic Choreography of Web Services: from WSDL to DAML-S, in the proceedings of International Conference on Web Services (ICWS 2003), June 2003.
  14. Fensel, D., Bussler, C., 2002. The web service modeling framework WSMF. Electronic Commerce Research and Applications, pages 209-231, 2002.
  15. Roman, D., Lausen, H. Keller, U., 2006. D2v1.3. Web Service Modelling Ontology (WSMO). Deliverable, http://www.wsmo.org/TR/d2/v1.3/, Oct 2006.
  16. Cimpian E., Mocan, A., Scharffe, F., Scicluna, J., Stollberg, M., 2005. D29v0.1 WSMO Mediators, WSMO Final Draft, December 2005, Available at: http://www.wsmo.org/TR/d29/v0.1/
  17. Martin-Recuerda, F., Sapkota, B., (eds.), 2005. WSMX Triple-Space Computing. Deliverable D21, 2005; available at: http://www.wsmo.org/TR/d21
  18. Moran, M., Polleres, A., Kopecký, J., WSMX Grounding, WSMX Working draft D26v0.1, December 2004. Available at http://www.wsmo.org/2004/d26/v0.1
  19. Sirin, E., Parsia, B., Cuenca Grau, B., Kalyanpur, A., Katz, Y., 2007. Pellet: A practical owl-dl reasoner. Journal of Web Semantics, June 2007.
  20. Akkiraju, R., Farrell, J., Miller, J., Nagarajan, M., Schmidt, M., Sheth, A., Verma, K., 2005. Web Service Semantics - WSDL-S. Technical note Available from http://lsdis.cs.uga.edu/ library/download/WSDL-S-V1.html, April 2005.
  21. Zhang, T., Ramakrishnan, R., Livny, M., 1997. BIRCH: A New Data Clustering Algorithm and Its Applications, Springer Journal on Data Mining and Knowledge Discovery, Vol 1, Issue 1, pp 141-182, June 1997.
  22. Shafiq, O., Alhajj, R., Rokne, J., 2014. Handling incomplete data using Semantic Logging based Social Network Analysis Hexagon for Effective Application Monitoring and Management, International Conference on Advances in Social Networks Analysis and Mining (IEEE/ACM ASONAM 2014), August 2014, Beijing, China.
  23. Shafiq, O., Alhajj, R., Rokne, J., Reducing Problem Space using Bayesian Classification on Semantic Logs for Enhanced Application Monitoring and Management, Intl Conf on Cognitive Informatics and Cognitive Computing (IEEE ICCI-CC 2014), August 2014, London, UK.
  24. Shafiq, O., Alhajj, R., Rokne, J., 2015. Reducing Search Space for Web Service Ranking using Semantic Logs and Semantic FP-Tree based Association Rule Mining, 9th IEEE Intl Conf on Semantic Computing (IEEE ICSC 2015), Feb 2015, Anaheim, CA, USA.
  25. Shafiq, O. M., 2016. Event Segmentation using MapReduce based Big Data Clustering, 2016 IEEE International Conference on Big Data (IEEE BigData 2016), December 2016, Washington, DC, USA.
Download


Paper Citation


in Harvard Style

Shafiq M. (2017). Integrated Analytics for Application Management using Stream Clustering and Semantics . In Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-247-9, pages 280-287. DOI: 10.5220/0006334802800287


in Bibtex Style

@conference{iceis17,
author={M. Omair Shafiq},
title={Integrated Analytics for Application Management using Stream Clustering and Semantics},
booktitle={Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2017},
pages={280-287},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006334802800287},
isbn={978-989-758-247-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - Integrated Analytics for Application Management using Stream Clustering and Semantics
SN - 978-989-758-247-9
AU - Shafiq M.
PY - 2017
SP - 280
EP - 287
DO - 10.5220/0006334802800287