Beyond Rules: How Large Language Models Are Redefining Cryptographic Misuse Detection

Zohaib Masood, Miguel Vargas Martin

2025

Abstract

The use of Large Language Models (LLMs) in software development is rapidly growing, with developers increasingly relying on these models for coding assistance, including security-critical tasks. Our work presents a comprehensive comparison between traditional static analysis tools for cryptographic API misuse detec-tion—CryptoGuard, CogniCrypt, and Snyk Code—and the LLMs—GPT, Llama, Claude, and Gemini. Using benchmark datasets (OWASP, CryptoAPI, and MASC), we evaluate the effectiveness of each tool in identifying cryptographic misuses. Our findings show that GPT 4-o-mini surpasses current state-of-the-art static analysis tools on the CryptoAPI and MASC datasets, though it lags on the OWASP dataset. Additionally, we assess the quality of LLM responses to determine which models provide actionable and accurate advice, giving developers insights into their practical utility for secure coding. This study highlights the comparative strengths and limitations of static analysis versus LLM-driven approaches, offering valuable insights into the evolving role of AI in advancing software security practices.

Paper Citation

in Harvard Style

Masood Z. and Martin M. (2025). Beyond Rules: How Large Language Models Are Redefining Cryptographic Misuse Detection. In Proceedings of the 22nd International Conference on Security and Cryptography - Volume 1: SECRYPT; ISBN 978-989-758-760-3, SciTePress, pages 179-194. DOI: 10.5220/0013524100003979

in Bibtex Style

@conference{secrypt25,
author={Zohaib Masood and Miguel Martin},
title={Beyond Rules: How Large Language Models Are Redefining Cryptographic Misuse Detection},
booktitle={Proceedings of the 22nd International Conference on Security and Cryptography - Volume 1: SECRYPT},
year={2025},
pages={179-194},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013524100003979},
isbn={978-989-758-760-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 22nd International Conference on Security and Cryptography - Volume 1: SECRYPT
TI - Beyond Rules: How Large Language Models Are Redefining Cryptographic Misuse Detection
SN - 978-989-758-760-3
AU - Masood Z.
AU - Martin M.
PY - 2025
SP - 179
EP - 194
DO - 10.5220/0013524100003979
PB - SciTePress