Insightor: AI-App Security Analysis
- Insightor is a large-scale security analysis framework that scans AI-powered applications on pre-trained model hubs for vulnerabilities like injection, secret leakage, and platform flaws.
- It employs a hybrid approach combining rule-based checks, taint/data-flow analysis, and external secret scanners to identify threats such as ghost tokens and over-privileged iframes.
- Corpus-scale evaluations on nearly one million AI-Apps demonstrate its capability to filter raw app data into actionable vulnerability candidates for manual validation.
Insightor is a large-scale security analysis framework for AI-powered applications hosted on pre-trained model hubs such as Hugging Face, Replicate, and ModelScope. It is designed to scan public AI-Apps at ecosystem scale and flag potential vulnerabilities arising from platform design flaws, application code flaws, runtime and configuration flaws, secret leakage, and malicious or poisoned AI-Apps. Rather than acting as a runtime protection system, it functions as a static-analysis-driven screening framework that combines rule-based checks for platform-specific weaknesses, taint and data-flow analysis for injection and leakage, and external scanners for secrets and malicious code. Its stated purpose is to reduce a corpus of nearly one million AI-Apps to a smaller set of actionable vulnerability candidates for manual validation (Gu et al., 29 Jun 2026).
1. Scope and problem setting
Insightor was developed from the observation that AI-Apps are not merely model artifacts but deployed, networked web services with authentication, logs, secrets, duplication, embedding, and container or runtime behavior. The paper argues that those platform-mediated features create security risks that traditional model-centric analyses miss. AI-Apps are described as cloud services wrapping pre-trained models with runtime environments, inference APIs, web UIs, secrets, training workflows, and billing controls, and the broader claim is that these features introduce a new attack surface because the apps are often created by mostly untrusted developers, deployed on shared platform infrastructure, exposed through browser UIs and APIs, and frequently granted access to sensitive runtime resources (Gu et al., 29 Jun 2026).
The paper models a typical AI-App lifecycle as follows: a developer creates an AI-App with a unique identifier; the developer either submits source code to be built by the platform, builds and pushes a container image, or duplicates an existing AI-App; the platform allocates runtime resources and launches the app; users access it via browser UI, embedded iframe, or API; and the platform provides update, deletion, and ownership-transfer functions. The study considers both public AI-Apps and private AI-Apps, assumes that platforms are trusted providers and that underlying channels such as HTTPS are secure, and explicitly excludes model-math-specific attacks and deployment-infrastructure attacks from scope (Gu et al., 29 Jun 2026).
Two adversary classes structure the threat model. Malicious developers may publish harmful AI-Apps, steal user data, phish, perform cryptojacking, distribute malicious container images, or inject code into third-party sites via platform weaknesses. Malicious users may exploit vulnerabilities in legitimate AI-Apps or platforms, exfiltrate logs and secrets, implant backdoors, or use reverse engineering and source inspection against deployed services. This framing positions Insightor as an ecosystem-level analysis tool rather than a single-application auditor (Gu et al., 29 Jun 2026).
2. Threat taxonomy and security framing
The paper identifies five threat categories and ten attack vectors, and maps them to OWASP Web Top 10, OWASP LLM Top 10, and supply-chain taxonomies. This is important because the study treats AI-App security as a hybrid of web application security, AI application security, and software supply-chain security rather than as an extension of model evaluation alone (Gu et al., 29 Jun 2026).
| Vector | Name | Brief characterization |
|---|---|---|
| A1 | Log Exfiltration | Unauthorized access to logs |
| A2 | Ghost Token | Token persistence plus namespace reuse |
| A3 | Authentication Bypass | Broken auth in app logic |
| A4 | Over-privileged iframe | Excessive browser permissions |
| R1 | Identifier Reuse | Dangling or colliding identifiers |
| R2 | Code Poisoning | Malicious inheritance or duplication |
| V1 | Input Injection | User input reaches execution sinks |
| L1 | Runtime Log Leakage | Secrets or sensitive data in logs |
| L2 | Files Leakage | Sensitive files exposed |
| P1 | Cryptojacking | Mining logic in AI-Apps |
Representative mappings illustrate the paper’s taxonomy strategy. A1 is mapped to OWASP Web A01 Broken Access Control and A05 Security Misconfiguration, and to OWASP LLM LLM02 Sensitive Information Disclosure and LLM07 System Prompt Leakage. A2 is mapped to A07 Identification and Authentication Failures, A01 Broken Access Control, and supply-chain category AV-501 Dangling Reference. V1 is mapped to A03 Injection, A06 Vulnerable and Outdated Components, and AV-300 Inject into Sources. R2 is mapped to A08 Software and Data Integrity Failures, LLM03 Supply Chain Vulnerabilities, and AV-100 Malicious Package. The paper uses these correspondences to argue that AI-App platforms amplify ordinary web and supply-chain flaws through platform-specific deployment and reuse mechanisms (Gu et al., 29 Jun 2026).
3. Detection pipeline and technical methods
Insightor is presented as a hybrid static-analysis and rule-based screening pipeline. Its high-level workflow is: collect AI-App corpora, including source repositories, container images, and metadata; apply heuristic and rule-based filters for platform-wide flaws such as ghost token and identifier reuse; run taint and data-flow analysis for input injection and sensitive-data leakage; run secret scanners; run malicious-code analysis; and manually validate sampled candidates. The paper explicitly characterizes Insightor as a high-efficiency filter rather than a perfect verifier (Gu et al., 29 Jun 2026).
For input injection analysis, Insightor performs CodeQL-based data-flow analysis on Python AI-Apps. The sources include predict(:str) and train(:str) in Replicate, Textbox() in Gradio, and text_input() and text_area() in Streamlit. Dangerous sinks include os.system, os.popen, eval, exec, and subprocess.*. The framework adapts generic taint analysis to AI-App SDKs by recognizing higher-level routing patterns that ordinary web taint rules miss. For template-based apps it treats template function parameters as sources; for Gradio and Streamlit apps it extracts UI widget inputs as sources. It additionally checks Gradio versions against OSV for CVE-2023-6572, CVE-2024-1540, CVE-2024-1728, CVE-2024-39236, and CVE-2024-47867 (Gu et al., 29 Jun 2026).
For data leakage analysis, the paper distinguishes several mechanisms. Hard-coded token leakage is detected by extracting all text files from repositories and images and then running KeySentinel secret scanning. JWT token leakage is studied through framework logging behavior, especially in Flask, FastAPI, Uvicorn, and Tornado, with the observation that query parameters containing JWTs often land in logs by default. Code-guided log leakage is detected by treating os.environ.get(), os.getenv(), and os.environb.get() as secret-retrieval sources, and print, logger.*, logging.*, and log.* as sinks, then using CodeQL to detect flows from secrets or user inputs into logs. Malicious code and poisoning analysis is performed by analyzing duplicated apps and training-generated code inheritance, using GuardDog to identify obvious malicious behaviors and then manually confirming suspicious backdoors and code-execution paths (Gu et al., 29 Jun 2026).
4. Corpus-scale measurement results
Insightor was applied to 972,546 public AI-Apps collected from September 2024 to December 2025: 938,602 on Hugging Face, 25,340 on Replicate, and 8,604 on ModelScope. The Hugging Face corpus included 713,613 downloadable container images, the Replicate corpus included 19,412 downloadable images, and the ModelScope corpus included 12 official template images in addition to the app set. The scale of this collection is central to the paper’s claim that AI-App vulnerabilities must be studied as ecosystem phenomena rather than isolated misconfigurations (Gu et al., 29 Jun 2026).
| Finding class | Measured result | Notes |
|---|---|---|
| Hard-coded token leakage | 3,418 unique leaked tokens across 4,846 AI-Apps | Frequent types include RSA private keys |
| Runtime log leakage | 936 Hugging Face apps leaked secrets in logs | 94 true leak cases in a 500-sample manual check |
| Input injection | 1,442 potential injection points detected | 83 confirmed in a 300-sample manual check |
| Gradio RCE exposure | 139,475 apps exposed to at least one known Gradio RCE | Version-based exposure metric |
| Multi-RCE exposure | 113,286 apps exposed to multiple RCEs | Version-based exposure metric |
| Code poisoning / backdoors | 27 apps with obvious malicious behaviors or backdoors | Found with GuardDog plus manual confirmation |
| Cryptojacking | 43 AI-Apps contained mining code | 42 on Hugging Face, 1 on Replicate |
| App duplication | 23,533 Hugging Face apps were duplicated | Relevant to poisoning propagation |
| Training-based inheritance | 1,327 Replicate cases inherited source code | Relevant to poisoning propagation |
Several platform-specific measurements are especially emphasized. On Hugging Face, 132,853 apps used 253,755 secrets, and 936 apps leaked secrets in logs; the leaked data included API keys, cloud tokens, prompts, system prompts, JWTs, and even medical data. On Replicate, 6,094 apps defined 11,525 secrets, but the paper reports no confirmed leakage among 3,772 apps with available logs. For authentication bypass, 65 AI-Apps were found with hard-coded passwords on Hugging Face and Replicate. For over-privileged iframes, 21,444 Hugging Face AI-Apps requested at least one browser permission, with common permissions including clipboard-write, autoplay, camera, encrypted-media, and microphone (Gu et al., 29 Jun 2026).
The paper also reports that 439,887 Hugging Face apps contain hyphens, and 438,967 of those have at least one unregistered username that could be exploited after deletion; over a 30-day observation window, 5,069 app deletions were observed. PublicWWW identified 172 apps embedded on 152 websites, and 9 websites were directly vulnerable to takeover via dangling iframe subdomains. These figures support the argument that identifier and embedding semantics can turn namespace management into a security-critical platform function (Gu et al., 29 Jun 2026).
5. Architectural vulnerabilities and security consequences
A major contribution of the paper is the identification of three novel architecture-induced vulnerabilities. The first is the Ghost Token attack on Hugging Face. The mechanism combines reusable user or account names, stateless JWTs, and an API endpoint that can generate JWTs for an app. Because tokens remain valid even after ownership changes, an attacker can predict a future target username or app identifier, create and delete the matching account and app, pre-generate a valid token, wait until a victim later claims the same identifier, and then use the old token to access the victim’s private app. The paper presents this as a serious authentication failure produced by the interaction between stateless token issuance and namespace reuse (Gu et al., 29 Jun 2026).
The second architectural issue is Identifier Reuse on Hugging Face. App subdomains are generated by replacing / with -, while usernames themselves may also contain hyphens, so distinct app identifiers can collide. If a third-party website embeds an app iframe and that app is later deleted, an attacker may create a different app that reclaims the dangling subdomain. The old iframe then loads attacker-controlled content from a trusted Hugging Face domain, enabling malicious code injection into third-party sites, phishing, credential theft, and takeover of embedded content (Gu et al., 29 Jun 2026).
The third is Over-privileged iframe behavior on Hugging Face and ModelScope. Hugging Face grants up to 27 browser permissions by default, including camera, microphone, clipboard-read and clipboard-write, geolocation, fullscreen, USB, and payment. The paper states that if a benign app has already been granted permissions, later malicious embedded apps can inherit them without new prompts, enabling stealthy access to sensors and clipboard data. This analysis is used to argue that browser embedding policy is not a peripheral UI detail but a core part of platform security design (Gu et al., 29 Jun 2026).
The practical consequences are not described as merely hypothetical. The paper reports credential leaks despite platform secret-management mechanisms, arbitrary code-execution paths through unsanitized inputs reaching os.system, eval, subprocess, and related sinks, and backdoors that functioned as C2 systems, remote-code-execution payloads, phishing mechanisms, credential and cookie theft tools, reconnaissance utilities, and hidden malware. Some malicious apps persisted for over a year and propagated through cloning or duplication, and the paper states that after disclosure, 8 were removed at the time of writing. The authors therefore argue that the AI-App ecosystem actively amplifies ordinary web and supply-chain bugs into platform-scale security failures (Gu et al., 29 Jun 2026).
6. Evaluation, limitations, and position in the insight literature
Insightor’s evaluation is centered on screening effectiveness rather than proof-style verification. For input injection detection, the paper manually reviewed 300 sampled flagged cases and confirmed 83 true positives, yielding 27.67% precision. For secret leakage in logs, it manually reviewed 500 sampled flagged cases and found 94 true leak cases, yielding 18.8% precision. To estimate false negatives, the authors focused on higher-risk subsets: 30,876 apps with secret variables and logging sinks for L1 risk, and 2,547 apps with code-execution sinks for V1 risk. From the remaining overlap, 879 candidates were sampled; manual review of 100 found 3 missed V1 cases and 2 missed L1 cases, which the paper interprets as indicating robust recall on the higher-risk segment (Gu et al., 29 Jun 2026).
The limitations are stated explicitly. The study focuses primarily on container-based AI-Apps because non-container AI-Apps are described as less customized and lower risk. Insightor is a screening tool with both false positives and false negatives, reflecting the standard tradeoff of static analysis at very large scale. The corpus is limited to public AI-Apps, so private organizational deployments are not fully studied. These caveats do not negate the findings, but they delimit the claims that can be made about prevalence and transferability across all deployment settings (Gu et al., 29 Jun 2026).
The paper’s defense guidance is correspondingly platform-oriented and operational. For platform designers it recommends enforcing least privilege, preventing unauthorized log exposure, adding token revocation for stateless authentication systems such as JWT, restricting iframe permissions, vetting AI-Apps with static analysis and malicious-code detection, preventing dangerous identifier reuse, preserving namespaces for deleted accounts, and masking secrets in logs automatically. For developers it recommends not hard-coding secrets or passwords, using platform secrets systems or dedicated secret managers such as AWS Secrets Manager or HashiCorp Vault, sanitizing inputs before passing them to shell or eval-like sinks, inspecting inherited code when duplicating AI-Apps, and treating logs as sensitive output. For users it recommends inspecting source code before using third-party AI-Apps, using private clones for sensitive tasks, and not assuming that a platform-hosted app is trustworthy merely because it appears on an official domain (Gu et al., 29 Jun 2026).
Within the wider literature on “insight” systems, Insightor occupies a distinct position. Foresight recommends statistically meaningful visual insights over an explicitly structured insight space for exploratory data analysis (Demiralp et al., 2017); DataSite surfaces background computational findings as a proactive feed in visual analytics (Cui et al., 2018); InsightPilot uses LLM-orchestrated intentional queries to drive automated data exploration (Ma et al., 2023); and DataSage treats insight discovery as an iterative multi-agent process with retrieval, debate, and code refinement (Liu et al., 18 Nov 2025). A formal synthesis of visualization theory defines an insight as a structured, higher-level knowledge object linking analytic knowledge with domain knowledge in a knowledge graph (Battle et al., 2022). This suggests that Insightor extends the “insight” label into security measurement: its outputs are grounded, evidence-backed findings over AI-App ecosystems rather than visual, statistical, or business-analytic observations. A separate 2026 paper also uses the name “Insightor” for a multi-agent corpus-level trace diagnostics system, referred to there as Insights Generator (IG), which produces evidence-backed reports about recurring patterns in LLM-agent execution traces (Manglik et al., 20 May 2026). The name therefore denotes a broader family of research systems centered on structured, corpus-level analytic findings, even though the underlying objects of analysis differ substantially.