Phishing Detection 2.0: A Natural Language Processing Approach to Identifying Generative AI-Crafted Social Engineering

Abstract

Phishing attacks have entered a new phase with the rise of generative artificial intelligence. Messages produced by large language models are often coherent, grammatically clean, and tailored to specific targets, which makes them harder to distinguish from authentic communication. Traditional phishing detection tools usually rely on rigid rule sets, surface-level text cues, or metadata analysis. These approaches work reasonably well on crude phishing attempts but fall short against AI-crafted content that mimics human writing patterns. This study presents an NLP-based approach aimed at identifying phishing messages generated by modern AI systems and separating them from both legitimate communication and human-written phishing attempts.The work begins by assembling a balanced dataset drawn from three sources: authentic organizational emails, verified human-written phishing samples, and synthetic phishing messages created with several generative models. The dataset is cleaned, standardized, and segmented to reduce noise and preserve linguistic features. The proposed system, Phishing Detection 2.0, uses a dual-channel feature extraction process. The first channel uses contextual embeddings from transformer models to capture high-level semantic relationships in the text. The second focuses on stylistic signatures that often differ between human and AI writing. These include perplexity, sentence rhythm, repetition patterns, lexical diversity, and entropy-based indicators. By combining semantic and stylistic signals, the system creates a richer representation of each message.The model architecture merges both channels through a fusion layer and passes the output to a classifier that predicts one of three categories: legitimate human communication, human-written phishing, or AI-generated phishing.

Topics & Concepts