Malware Detection for Visualized Images Using Hybrid Fast R-CNN and Transformation Models

Swathi Anil, Ananya S Mallia, Manazhy Rashmi

2025

Abstract

Malware visualization techniques are becoming increasingly sophisticated, posing significant challenges to traditional detection systems. To address this, we propose a novel Hybrid Fast R-CNN and Transformation Model (HFRTM) framework for the accurate detection and classification of malware in visualized RGB images and malicious network behaviours. The HFRTM integrates Fast Region-based Convolutional Neural Networks (Fast R-CNN) for efficient malware pattern detection and localization with transformation models to enhance feature extraction by capturing complex variations in malware appearances. Key enhancements like fine-tuning of transformation models on a specialized target dataset, leveraging pre-trained weights to accelerate convergence and mitigate overfitting. This ensemble architecture demonstrates superior accuracy and robustness, effectively distinguishing malware from benign data even in challenging scenarios. To validate the efficacy of HFRTM, extensive experiments were conducted on a benchmark malware visualization dataset. The proposed method achieved a detection accuracy of 98.7%, significantly outperforming existing state-of-the-art methods. The results highlight the practical applicability of HFRTM in real-world cybersecurity scenarios, offering an advanced and reliable solution for combating sophisticated malware threats.

Paper Citation

in Harvard Style

Anil S., S Mallia A. and Rashmi M. (2025). Malware Detection for Visualized Images Using Hybrid Fast R-CNN and Transformation Models. In Proceedings of the 3rd International Conference on Futuristic Technology - Volume 1: INCOFT; ISBN 978-989-758-763-4, SciTePress, pages 574-581. DOI: 10.5220/0013582000004664

in Bibtex Style

@conference{incoft25,
author={Swathi Anil and Ananya S Mallia and Manazhy Rashmi},
title={Malware Detection for Visualized Images Using Hybrid Fast R-CNN and Transformation Models},
booktitle={Proceedings of the 3rd International Conference on Futuristic Technology - Volume 1: INCOFT},
year={2025},
pages={574-581},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013582000004664},
isbn={978-989-758-763-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Conference on Futuristic Technology - Volume 1: INCOFT
TI - Malware Detection for Visualized Images Using Hybrid Fast R-CNN and Transformation Models
SN - 978-989-758-763-4
AU - Anil S.
AU - S Mallia A.
AU - Rashmi M.
PY - 2025
SP - 574
EP - 581
DO - 10.5220/0013582000004664
PB - SciTePress