Enhancing Security Event Management Systems with Unsupervised Anomaly Detection

Markus Goldstein, Stefan Asanger, Matthias Reif, Andrew Hutchison

2013

Abstract

Security Information and Event Management (SIEM) systems are today a key component of complex enterprise networks. They usually aggregate and correlate events from different machines and perform a rule-based analysis to detect threats. In this paper we present an enhancement of such systems which makes use of unsupervised anomaly detection algorithms without the need for any prior training of the system. For data acquisition, events are exported from an existing SIEM appliance, parsed, unified and preprocessed to fit the requirements of unsupervised anomaly detection algorithms. Six different algorithms are evaluated qualitatively and finally a global k-NN approach was selected for a practical deployment. The new system was able to detect misconfigurations and gave the security operation center team more insight about processes in the network.

References

  1. Albayrak, S. and Wieczorek, D. (1998). JIAC - an open and scalable agent architecture for telecommunication applications. Intelligent Agents for Telecommunication Appl.
  2. Amer, M. and Goldstein, M. (2012). Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer. In Proc. of the 3rd RCOMM 2012.
  3. Angiulli, F. and Pizzuti, C. (2002). Fast outlier detection in high dimensional spaces. In PKDD, volume 2431 of LNCS, pages 43-78. Springer.
  4. Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. (2000). Lof: identifying density-based local outliers. SIGMOD Rec., 29(2):93-104.
  5. Buecker, A., e. a. (2008). IBM Tivoli Security Operations Manager 4.1. Deployment Guide Series. IBM Redbooks, 1st edition.
  6. Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM Comput. Surv., 41(3):1-58.
  7. Endsley, M. R. (1987). The application of human factors to the development of expert systems for advanced cockpits. In Human Factors Society 31st Annual Meeting.
  8. Garcia-Teodoro, P. and et al (2009). Anomaly-based network intrusion detection: Techniques, systems and challenges. Computers and Security, 28:18 - 28.
  9. Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11:1-21.
  10. He, Z., Xu, X., and Deng, S. (2003). Discovering clusterbased local outliers. Pattern Recognition Letters, 24(9-10):1641 - 1650.
  11. Hodge, V. J. and Austin, J. (2004). A survey of outlier detection methodologies. Artificial Intelligence Rev., 22.
  12. IBM (2008). Tivoli Security Operations Manager Version 4.1.1 - Administration Guide. IBM, 1st edition. http://publib.boulder.ibm.com/infocenter/tivihelp/v2r 1/topic/com.ibm.netcool som.doc/tsom411 admin.pdf.
  13. Jin, W., Tung, A., Han, J., and Wang, W. (2006). Ranking outliers using symmetric neighborhood relationship. In Advances in Knowledge Discovery and Data Mining, volume 3918 of LNCS, pages 577-593. Springer.
  14. Kriegel, H.-P., Kröger, P., Schubert, E., and Zimek, A. (2009). Loop: local outlier probabilities. In CIKM 7809, pages 1649-1652. ACM.
  15. Mierswa, I., Wurst, M., and et al (2006). Yale (now: Rapidminer): Rapid prototyping for complex data mining tasks. In Proc. of the ACM SIGKDD 2006.
  16. Miller, D. R. e. a. (2011). Security Information and Event Management (SIEM) Implementation. The McGrawHill Companies.
  17. Nicolett, M., e. a. (2011). Magic Quadrant for Security Information and Event Management. Gartner RAS Core Research, (ID:G00212454).
  18. Nicolett, M. and Kavanagh, K. M. (2012a). Critical Capabilities for Security Information and Event Management. Gartner RAS Core Research, (ID:G00227900).
  19. Nicolett, M. and Kavanagh, K. M. (2012b). Magic Quadrant for Security Information and Event Management. Gartner RAS Core Research, (ID:G00227899).
  20. Ramaswamy, S., Rastogi, R., and Shim, K. (2000). Efficient algorithms for mining outliers from large data sets. In Proce. of the 2000 ACM SIGMOD, pages ”427-438”.
  21. Rodriguez, A. C. and de los Mozos, M. R. (2010). Improving network security through traffic log anomaly detection using time series analysis. In CISIS'10, pages 125-133.
  22. Tang, J., Chen, Z., Fu, A., and Cheung, D. (2002). Enhancing effectiveness of outlier detections for low density patterns. In Advances in Knowledge Discovery and Data Mining, volume 2336 of LNCS, pages 535-548. Springer.
Download


Paper Citation


in Harvard Style

Goldstein M., Asanger S., Reif M. and Hutchison A. (2013). Enhancing Security Event Management Systems with Unsupervised Anomaly Detection . In Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-8565-41-9, pages 530-538. DOI: 10.5220/0004230105300538


in Bibtex Style

@conference{icpram13,
author={Markus Goldstein and Stefan Asanger and Matthias Reif and Andrew Hutchison},
title={Enhancing Security Event Management Systems with Unsupervised Anomaly Detection},
booktitle={Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2013},
pages={530-538},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004230105300538},
isbn={978-989-8565-41-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Enhancing Security Event Management Systems with Unsupervised Anomaly Detection
SN - 978-989-8565-41-9
AU - Goldstein M.
AU - Asanger S.
AU - Reif M.
AU - Hutchison A.
PY - 2013
SP - 530
EP - 538
DO - 10.5220/0004230105300538