PhishParrot: Adaptive Crawling for Phishing

• The paper presents PhishParrot as an adaptive crawling system that uses LLM-driven profile generation to counter cloaked phishing by tailoring the crawling environment.
• It employs a four-module architecture combining feature gathering, vector retrieval with cosine similarity and MMR, and JSON-based user profile synthesis via an LLM.
• Evaluation on 32,487 URLs shows PhishParrot uncovers hidden phishing artifacts by synthesizing 91 optimal environments, significantly outperforming standard crawlers.

PhishParrot is an LAG-driven adaptive crawling environment optimization system for detecting cloaked phishing sites. In the described threat model, cloaking allows attackers to present phishing content only to users who satisfy attacker-specified criteria while returning benign pages or HTTP errors to security crawlers. PhishParrot addresses this asymmetry by collecting multi-environment crawling information, retrieving semantically similar successful and failed past crawls, and using a LLM to generate an optimized user profile for re-crawling. The resulting environment is intended to match the attacker’s expected victim conditions closely enough to reveal the hidden phishing payload. In a 21-day evaluation, the system improved detection accuracy by up to 33.8% over standard analysis systems and yielded 91 distinct crawling environments for diverse conditions targeted by attackers (Nakano et al., 4 Aug 2025).

1. Cloaked Phishing as the Operational Context

Cloaking in phishing is the practice of serving benign or error pages to security crawlers while delivering credential-stealing content only to users who match attacker-specified criteria. The checks described for this setting include HTTP headers such as User-Agent and Accept-Language, source IP attributes such as ASN and geolocation, and behavioral signals such as mouse or keyboard activity. If any check fails, the observed page may be a fake “legitimate” page or an HTTP error rather than the phishing form itself (Nakano et al., 4 Aug 2025).

This setting creates a structural failure mode for conventional phishing analysis pipelines. A conventional crawler typically assumes a single, static environment, exemplified in the paper by headless Chrome running from a U.S. datacenter. Under cloaking, such a crawler can systematically fail environmental checks and therefore never observe the malicious payload. When the phishing form is absent from the retrieved content, downstream machine-learning classifiers or rule-based engines cannot flag the site as malicious. PhishParrot is positioned specifically as a remedy for this environment-mismatch problem rather than as a standalone classifier.

A plausible implication is that PhishParrot is best understood as an acquisition-layer system: its primary function is to recover the hidden artifacts that existing detectors require but static crawlers often fail to obtain.

2. Four-Module Architecture and System Role of the LLM

PhishParrot consists of four sequential modules: Preliminary Access & Feature Gathering, Similar-Case Retrieval via Vector Database, LLM-Driven Profile Generation, and Adaptive Crawling & Final Extraction (Nakano et al., 4 Aug 2025).

The pipeline begins with a suspicious URL. Under a default environment, the system collects domain, network, and HTML snippets. These features are then embedded into a vector database, which is queried to retrieve semantically similar past crawls, including both successful and failed cases. The retrieved history is assembled into a prompt for an LLM, with the paper giving GPT-4o mini as an example. The model is instructed as a “cybersecurity analyst” and produces a JSON user profile containing optimal HTTP headers, IP location, network provider, and a rationale. Finally, the URL is re-crawled under the recommended environment using Playwright together with proxies or VPNs to emulate geolocation and ASN, and the system extracts hidden phishing content, including HTML, screenshots, and network logs (Nakano et al., 4 Aug 2025).

The LLM’s function is not merely generative in a stylistic sense; it is used for contextual analysis over retrieved examples. The paper characterizes this as enabling inference of non-trivial patterns, such as an attacker preferring Japanese residential IPs, rejecting cloud datacenter ASNs, and targeting a recent Android Chrome user agent. The formal framing given for this stage is Retrieval-Augmented Generation (RAG).

A plausible misconception is that the LLM itself performs the final phishing decision. In the described architecture, the output is instead raw phishing artifacts fed to downstream phishing detectors.

3. Information Model, Storage, and Similar-Case Retrieval

PhishParrot structures each crawl into four categories of information:

• Domain Information: WHOIS registration, registrar, DNS answers such as A/AAAA records, and TLS certificate issuer or subject.
• Network Information: full HTTP request-cycle logs, including requests, responses, headers, and status codes.
• HTML Information: visible text snapshot and DOM tag structure skeleton.
• Crawling Environment Information: IP geolocation at the country, region, and city levels, ASN metadata including provider name and ASN number, and browser language setting (Nakano et al., 4 Aug 2025).

All of these are serialized as JSON and inserted into a vector database using 1,536-dimensional embeddings from OpenAI’s text-embedding-3-small. Each record is labeled either successful, meaning phishing content was obtained, or failed, meaning the site remained cloaked. This labeling is central to the later profile-construction stage because the prompt includes both positive and negative precedents rather than only successful examples.

Similarity matching is performed over embeddings for the three feature groups—domain, network, and HTML—using cosine similarity. Given two vectors $u,v\in\mathbb{R}^d$,

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$

Past records are retrieved when similarity is at least the threshold $\tau$, with $\tau=0.65$ (Nakano et al., 4 Aug 2025).

To improve representativeness, PhishParrot then applies Maximum Marginal Relevance (MMR) to choose $K$ examples that balance relevance and diversity. The objective is described as selecting $S\subset \mathrm{Candidates}$, $|S|=K$, to maximize

$$\max_{i\in \mathrm{Candidates}\setminus S}\Bigl[ \lambda\cdot \mathrm{Sim}(i,q) - (1-\lambda)\cdot \max_{j\in S}\mathrm{Sim}(i,j) \Bigr],$$

where $q$ is the query embedding, $\mathrm{Sim}$ is cosine similarity, and $$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$0 balances relevance versus diversity. The system packages up to 5 successful and 5 failed examples into the LLM prompt, filtered to essential fields.

This representation-and-retrieval design suggests that PhishParrot treats cloaking not as a purely per-URL phenomenon but as a pattern that can recur across related campaigns, infrastructure, or environmental targeting strategies.

4. User-Profile Construction and Adaptive Re-Crawling

The LLM is asked to output a JSON user profile of the form:

$\tau$4

This profile is the mechanism by which contextual retrieval is translated into an executable crawl environment. The generated fields directly control browser and network configuration, including headers and IP-level presentation. The paper describes the final crawl as being executed via Playwright with proxies or VPNs to emulate the recommended geolocation and ASN, after which phishing artifacts are extracted (Nakano et al., 4 Aug 2025).

The workflow is also expressed procedurally:

$\tau$5

The same process is summarized in the paper as a flow in words: start with default crawl, extract features, query the vector database, retrieve and compress history, feed the LLM, receive a tailored environment, re-crawl under that environment, and produce phishing content for detectors.

This workflow clarifies that PhishParrot is adaptive rather than brute-force. It does not enumerate all possible browsing environments blindly; it narrows the search using semantically similar prior cases and then lets the LLM synthesize a targeted victim profile.

5. Evaluation Design, Metrics, and Quantitative Results

The evaluation uses 32,487 live suspicious URLs over 21 days, with 15,309 phishing and 17,178 benign URLs according to VirusTotal consensus. Two baselines are defined: the Standard Analysis System, described as headless Chrome in a U.S. datacenter, and the Typical User System, described as a random pick among 510 http_header×location×ASN combos. The downstream detectors are ChatPhishDetector, VisualPhishnet, and StackModel (Nakano et al., 4 Aug 2025).

The reported metrics are:

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$1

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$2

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$3

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$4

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$5

In addition to these classification metrics, the evaluation tracks Execution Time / URL and LLM API Cost. Relative improvement from a baseline accuracy $$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$6 to a PhishParrot accuracy $$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$7 is defined as

$$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$8

The paper provides the example of ChatPhishDetector, where accuracy rises from 58.9% to 92.7%, yielding $$\mathrm{cosine\_similarity}(u,v)=\frac{u\cdot v}{\|u\|\;\|v\|}.$$9 (Nakano et al., 4 Aug 2025).

For statistical significance, the study applies a two-proportion z-test:

$\tau$0

with

$\tau$1

The reported result is that, in all comparisons of PhishParrot versus baselines, $\tau$2.

The key empirical findings are concentrated in three claims. First, GPT-4o mini is reported as the best LLM, with 88.0% average accuracy, 16.7 s/URL, and $0.002 cost. Second, PhishParrot+ChatPhishDetector achieves 92.7% accuracy, compared with 58.9% for the standard system and 64.4% for the typical-user system, corresponding to up to 33.8% absolute gain. Third, the system suggests 91 distinct optimal environments, with top profiles concentrating on Japanese residential Chrome users (36.8%) and U.S. datacenter Chrome (16.9%) (Nakano et al., 4 Aug 2025).

These results suggest that the principal performance gain arises from improved content acquisition under cloaking, not from altering the downstream detectors themselves.

6. Limitations, Attacker Counter-Defenses, and Future Directions

The paper identifies two direct limitations. First, profile selection adds 5–7 s overhead. Second, the method requires an initial seed of ~1,000 labeled examples to populate the vector DB (Nakano et al., 4 Aug 2025). These constraints indicate that the system depends both on prior case accumulation and on an additional inference step that is not free in latency or cost.

Potential attacker counter-defenses are also enumerated. These include more dynamic checks, such as behavioral biometrics and challenge–response CAPTCHAs; fingerprint flakiness detection, such as measuring rendering differences in headless versus headful execution; and time-based cloaking, in which benign content is served only for $\tau$3 seconds before switching. Each of these mechanisms attacks a different assumption in PhishParrot’s current workflow: the sufficiency of environment matching, the transparency of browser emulation, and the stability of content over time (Nakano et al., 4 Aug 2025).

The future directions listed are correspondingly operational. They include integrating active interaction such as form-filling and mouse events to defeat behavioral cloaks; incorporating network-level fuzzing by varying TTL, MTU, and HTTP/2 vs. HTTP/1.1; applying self-supervised retraining of embedding models on phishing-specific corpora; and using a real-time multi-armed bandit for profile testing to reduce LLM cost.

Taken together, these limitations and extensions position PhishParrot as a system for adaptive environment synthesis rather than a complete solution to all forms of cloaking. A plausible implication is that future phishing-crawling systems may need to combine environmental adaptation, interactive behavior synthesis, and transport-level variation within a unified acquisition stack.