Can AI Keep You Safe? A Study of Large Language Models for Phishing Detection
Robin Chataut, Prashnna Gyawali, Yusuf Usman
Abstract
Phishing attacks continue to be a pervasive challenge in cybersecurity, with threat actors constantly developing new strategies to penetrate email inboxes and compromise sensitive data. In this study, we investigate the effectiveness of Large Language Models (LLMs) in the crucial task of phishing email detection. With the growing sophistication of these attacks, we assess the performance of three distinct LLMs: GPT-3.5, GPT-4, and a customized ChatGPT, against a carefully curated dataset containing both phishing and legitimate emails. Our research reveals the proficiency of LLMs in identifying phishing emails, with each model showing varying levels of success. The paper outlines the strengths and limitations of GPT-3.5, GPT-4, and the custom ChatGPT, illuminating their respective suitability for practical applications in email security. These results underscore the potential of LLMs in effectively identifying phishing emails and their significant implications for enhancing cybersecurity measures and safeguarding users from the risks of online fraud.