A Self-Knowledge Distillation Approach to the Robustly Optimized BERT Approach for Common Vulnerabilities and Exposures

Gurinder Pal Singh, Rohit Bajaj, Manish Kumar Hooda

2025

Abstract

The ever-increasing threat of cyber assaults on critical infrastructure businesses has prompted a focus on bolstering their cyber security expertise. Common Vulnerabilities and Exposures (CV&E) are the most crucial to understand since they are a collection of flaws that may be discovered in a wide range of software and hardware. However, many vulnerabilities remain unaddressed, making it impossible for an attacker to exploit them against you. A well-known approach for managing cyber security risks called MITRE ATT&CK (Adversary Tactics, Techniques, and Common Knowledge) provides mitigation techniques for a variety of destructive tactics employed by adversaries. Despite the enormous advantages of ATT&CK and CVEs, cyber security stakeholders might benefit from this approach. The CVE model proposed in this study contains a self-knowledge distillation design applied to the pre-trained language model of the highest-caliber model, Robustly Optimised BERT Approach (RoBERTa). A proposed novelty is based on a high-quality dataset that can improve the model's F1-score. The proposed model exceeded the F1 score of 77.20% and improved the accuracy of 75.92% compared to conventional machine learning models. This study's findings the preliminary information from MITRE ATT&CK may be beneficial to cybersecurity stakeholders.

Paper Citation

in Harvard Style

Singh G., Bajaj R. and Hooda M. (2025). A Self-Knowledge Distillation Approach to the Robustly Optimized BERT Approach for Common Vulnerabilities and Exposures. In Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT; ISBN 978-989-758-763-4, SciTePress, pages 636-643. DOI: 10.5220/0013634900004664

in Bibtex Style

@conference{incoft25,
author={Gurinder Pal Singh and Rohit Bajaj and Manish Kumar Hooda},
title={A Self-Knowledge Distillation Approach to the Robustly Optimized BERT Approach for Common Vulnerabilities and Exposures},
booktitle={Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT},
year={2025},
pages={636-643},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013634900004664},
isbn={978-989-758-763-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT
TI - A Self-Knowledge Distillation Approach to the Robustly Optimized BERT Approach for Common Vulnerabilities and Exposures
SN - 978-989-758-763-4
AU - Singh G.
AU - Bajaj R.
AU - Hooda M.
PY - 2025
SP - 636
EP - 643
DO - 10.5220/0013634900004664
PB - SciTePress