Authors:
Nadira Anjum Nipa
;
Nizar Bouguila
and
Zachary Patterson
Affiliation:
Concordia Institute for Information and Systems Engineering, Concordia University, Montreal, Quebec, Canada
Keyword(s):
Anomaly Detection, Log Analysis, Machine Learning, Deep Learning, Log Parser.
Abstract:
The reliability and security of today’s smart and autonomous systems increasingly rely on effective anomaly detection capabilities. Logs generated by intelligent devices during runtime offer valuable insights for monitoring and troubleshooting. Nonetheless, the enormous quantity and complexity of logs produced by contemporary systems render manual anomaly inspection impractical, error-prone, and laborious. In response to this, a variety of automated methods for log-based anomaly detection have been developed. However, many current methods are evaluated in controlled environments with set assumptions and frequently depend on publicly available datasets. In contrast, real-world system logs present greater complexity, lack of labels, and noise, creating substantial challenges when applying these methods directly in industrial settings. This work explores and adapts existing machine learning and deep learning techniques for anomaly detection to function on real-world system logs produced
by an intelligent autonomous display device. We conduct a comparative analysis of these methods, evaluating their effectiveness in detecting anomalies through various metrics and efficiency measures. Our findings emphasize the most efficient approach for detecting anomalies within this specific system, enabling proactive maintenance and enhancing overall system reliability. Our work provides valuable insights and directions for adopting log-based anomaly detection models in future research, particularly in industrial applications.
(More)