EliBadCode: Understanding Bad Code in AI Systems
- EliBadCode is a research term that defines both the risks of blindly trusting AI-generated code and innovative frameworks for detecting and eliminating backdoors in neural code models.
- It integrates static and dynamic validation, trigger inversion, and attribution-based methods to identify and repair issues ranging from dead code to unsafe exception handling.
- Empirical results reveal that while LLM-generated code may seem functional, it exhibits significant security and robustness flaws, underscoring the need for rigorous testing and model hardening.
Searching arXiv for the specified papers and topic to ground the article with current bibliographic context. EliBadCode is a research term that appears in recent software-engineering and code-intelligence literature in more than one sense. In one usage, it is a shorthand for the danger of naively trusting AI-generated code, especially when code produced by LLMs is deployed without rigorous verification. In another, it is the name of a backdoor-elimination framework for neural code models, designed to reverse-engineer identifier-based triggers and unlearn malicious behavior. A broader interpretation, explicitly suggested in later work on dead-code elimination, treats “EliBadCode” as the general task of detecting, explaining, and correcting problematic code, including dead code, unsafe exception handling, language-specific smells, and semantically subtle vulnerabilities across large repositories (Chong et al., 2024, Sun et al., 2024, Chen et al., 4 Jun 2025).
1. Terminological scope and conceptual frame
Across the cited works, EliBadCode does not denote a single universally fixed artifact. The term is used explicitly as a shorthand in a study of AI-generated C code, where it names the danger of blindly trusting model outputs, and as a formal framework in work on securing neural code models against backdoor attacks (Chong et al., 2024, Sun et al., 2024). A broader interpretive extension appears in work on dead-code elimination, where the query “EliBadCode” is treated as the idea of a system that can eliminate bad code by locating, explaining, and repairing problematic patterns (Chen et al., 4 Jun 2025).
This dual use is significant because it spans two layers of the software stack. One layer concerns code artifacts themselves: incorrect implementations, insecure memory behavior, dead branches, swallowed exceptions, and code smells. The other concerns the models and tools that analyze or generate code: code LLMs that emit brittle implementations, or neural code models that can be backdoored. Taken together, the literature suggests that “bad code” is not merely stylistic deficiency. It includes semantic incorrectness, exploitable memory behavior, omitted defensive programming, poisoned model behavior, and structurally misleading code that interferes with downstream analysis.
A plausible implication is that EliBadCode is best understood not as one method but as a family of research problems at the intersection of secure code generation, program analysis, automated repair, and trustworthy code intelligence.
2. AI-generated code considered harmful
In the usage introduced by the secure-code-generation study, EliBadCode denotes the risk of treating LLM-generated code as trustworthy by default. That study evaluates C code generated by OpenAI GPT-4o, using an evaluation framework called EXACT (EXamination System for AI-Generated Code Testing) across three task classes: 202 LeetCode-style problems, data structures and algorithms drawn from repositories including “The Algorithms,” “fragglet,” and “petewarden,” and cryptographic routines including SHA-1, MD5, and AES from OpenBSD (Chong et al., 2024).
EXACT combines code generation, functionality validation, security analysis, and complexity/size measurement. Functionality validation uses the LeetCode online judge for interview-style tasks, CUnit and AFL for DS&A code, and NSRL crypto test vectors for cryptographic routines. Security analysis uses the Clang static analyzer with default plus “experimental” core, security, and Unix checkers. Complexity and size are measured using SCC, including cyclomatic complexity, lines of code, and complexity per line (Chong et al., 2024).
The central empirical result is that GPT-4o-generated code is frequently functional enough to appear useful, yet systematically less robust and less secure than human-written code. On the 202 LeetCode problems, 87.6% of GPT-4o solutions passed, while 12.4% failed. Among the failures, 9 were due to failing test cases or Time Limit Exceeded, 12 were runtime errors with sanitizers enabled, and 4 were compile errors caused by wrong function signatures. The runtime errors included heap buffer overflow, stack buffer overflow, array index out of bounds, signed integer overflow, load of address with insufficient space, and requested allocation size exceeds maximum supported size. Crucially, 8 of these 12 runtime errors were not detected by Clang static analysis, indicating that static analysis alone was insufficient to surface serious faults (Chong et al., 2024).
The paper reports that difficulty matters. Of the 21 LeetCode tasks GPT-4o failed, 20 were labeled medium or hard. In the DS&A setting, GPT-4o versions of 15 tasks passed unit tests, whereas the human code had 3/15 unit-test failures; however, fuzzing reversed the picture of reliability. GPT-4o code produced 12 unique hangs versus 6 for human code, and 451 unique crashes versus 364, yielding 50% more hangs and 23.9% more crashes. The conclusion was not that human code was flawless, but that LLM code was systematically more fragile under adversarial or unexpected inputs (Chong et al., 2024).
The cryptographic results are especially consequential. GPT-4o implementations of MD5 and AES produced correct outputs on all test vectors, but the SHA-1 implementation produced incorrect hash values for all inputs, despite compiling and running. The error was therefore silent and semantic rather than syntactic. In security-critical code, that distinction is decisive: a routine may appear structurally valid while violating the intended cryptographic specification (Chong et al., 2024).
The study also isolates characteristic failure modes. A toy buffer-size task shows that GPT-4o often reasons by pattern rather than by actual memory-safety requirements: in an alphabet string missing the letter G, the correct size is 25, yet the model often filled 26, causing heap overflow. Across 1000 trials per definition, success rates ranged from 99.3% for a constant integer, to 41.6% for the missing-letter alphabet case, to 1.5% for float × float, with rare outputs of gibberish non-English characters instead of numbers. This suggests unreliable numeric reasoning in contexts where exact arithmetic is required for memory safety (Chong et al., 2024).
The security delta also appears in static analysis. Over 220 files (202 LeetCode + 18 DS&A), GPT-4o code incurred 128 Clang issues versus 116 for human code, or 10.3% more static-analysis issues overall. The largest disparity was malloc overflow, with 81 issues for GPT-4o and 61 for human code, or 32.8% more in LLM-generated code. Meanwhile, GPT-4o code was shorter but denser: mean LoC was 23.6 versus 29.6 for LeetCode and 103.5 versus 192.1 for DS&A, yet complexity per line was higher for GPT-4o in both settings, 0.33 versus 0.28 for LeetCode and 0.21 versus 0.17 for DS&A. The paper characterizes this output as “bare-bones”: compressed logic with fewer explicit checks, fewer guardrails, and more brittle semantics (Chong et al., 2024).
Finally, the study shows that prompt-based repair is unreliable. A feedback loop over 60 GPT-4o LeetCode solutions reduced some vulnerabilities but introduced others. In files that already had issues, total targeted issues dropped from 58 to 44 after one regeneration loop, but array index out of bounds increased from 15 to 18. In 30 files initially free of the targeted Clang issues, regeneration introduced 4 new malloc overflow issues. Ancillary prompting experiments were also non-monotonic: asking for shorter line lengths yielded 11.3% lower cyclomatic complexity per line and fixed 17.9% of Clang issues, whereas asking for fewer lines of code made code 18.4% more complex per line but fixed 43.2% of Clang issues. No simple prompt recipe emerged as a reliable security control (Chong et al., 2024).
3. EliBadCode as backdoor elimination for neural code models
A distinct meaning of EliBadCode appears in work on neural code models used for defect detection, clone detection, and code search. Here EliBadCode is a defense framework for eliminating backdoors implemented through identifier renaming triggers. The threat model assumes a defender with white-box access to the model, access to a small clean dataset, and no access to the original poisoned training set or true trigger. The attacker poisons training data so that the model behaves normally on clean code but flips to an attacker-chosen output when code contains a stealthy identifier trigger (Sun et al., 2024).
The framework is organized around trigger inversion followed by unlearning. It first filters the model vocabulary according to programming-language naming rules, reducing the trigger search space to tokens that plausibly occur in identifiers. For CodeBERT, this reduces the vocabulary from 50,265 tokens to 15,838 trigger candidates. It then identifies sample-specific trigger positions by masking identifier positions and measuring the loss difference induced by masking. The least sensitive position is selected for each sample, on the empirical premise that true backdoor triggers are more position-robust than ordinary adversarial perturbations (Sun et al., 2024).
Trigger inversion is then performed with a Greedy Coordinate Gradient procedure over discrete token sequences. For a candidate target label , the framework minimizes an inversion loss of the form
where is the trigger, is a masked clean subset, and are the backdoored parameters. The label for which inversion yields the lowest loss is treated as the backdoor target. EliBadCode then applies trigger anchoring, removing tokens whose deletion causes only small loss change; with threshold , effective trigger tokens are separated from noise tokens (Sun et al., 2024).
Backdoor elimination is performed by model unlearning. The anchored trigger is injected into a fraction of clean samples, and only the last layer parameters are fine-tuned, rather than the entire model, to avoid catastrophic forgetting. The paper reports that a trigger injection rate of roughly 20% is sufficient to suppress attack behavior while preserving normal performance (Sun et al., 2024).
Evaluation covers CodeBERT, CodeT5, and UniXcoder on Devign for defect detection, BigCloneSearch for clone detection, and CSN-Python for code search. Two attacks are considered: CodePoisoner, using trigger “testo_init”, and BadCode, using trigger “rb”. Under CodePoisoner, average ASR drops from 99.04% to 0.93% for defect detection and from 100% to 4.73% for clone detection, while code-search ANR rises from 9.55 to 24.95; clean ACC on defect detection changes from 64.14% to 63.40%, and clone-detection F1 changes from 94.15% to 96.70%. Under BadCode, average ASR drops from 99.76% to 2.64% for defect detection and from 100% to 6.11% for clone detection, while code-search ANR rises from 9.99 to 25.18. A baseline defense, DBS, is markedly less effective in these settings (Sun et al., 2024).
The ablation results clarify which components are essential. Without PL-specific vocabulary filtering, trigger inversion fails and post-defense ASR remains 100%. Removing sample-specific position identification leaves end performance largely unchanged but increases inversion cost from roughly 25 epochs to 52 epochs. Removing trigger anchoring can push ASR even lower, but at a notable cost in clean accuracy: in one CodeBERT defect-detection setting, ACC falls to 60.98% instead of 62.57% with anchoring. The framework also requires enough clean data: with fewer than 20 clean samples, inversion fails, whereas 30 samples per class gave the best reported performance (Sun et al., 2024).
In this sense, EliBadCode is not about generated source code directly. It is a model hardening technique for securing code-understanding systems themselves.
4. Related efforts to detect, explain, and repair problematic code
A broader “eliminate bad code” interpretation is made explicit in work on dead code elimination. DCE-LLM defines dead code as unused or unreachable code and frames it as both a software-engineering and a security problem, including its role in malware & obfuscation and in LLM vulnerabilities such as dead-code injection attacks. The system combines a CodeBERT-based pivot model with attribution-based line selection and a fine-tuned Qwen2-7B-Instruct expert model that outputs a structured judgment, line numbers, type labels, explanations, and fixed code. It is trained on AIDCE, a dataset of 19,639 samples from CodeNet with 12,628 normal, 4,495 unused, and 2,516 unreachable samples, plus an SFT dataset of approximately 1,500 samples using GPT-4o-generated silver explanations and fixes (Chen et al., 4 Jun 2025).
Its performance is substantially above generic LLM baselines. On the AIDCE test split, DCE-LLM achieves F1 = 94.02 for unused, 96.65 for unreachable, and 99.38 for normal, with overall accuracy = 96.40. By contrast, GPT-4o (zero-shot) achieves F1 = 40.98 for unused, 64.00 for unreachable, 76.96 for normal, and accuracy = 60.97. The paper summarizes this as over 94% F1 on unused and unreachable code and a performance margin of more than 30% F1 over GPT-4o on dead-code classes. In an unseen Golang setting, DCE-LLM still reaches F1 = 90.42 for unused and 90.91 for unreachable, compared with 67.59 and 81.87 for GPT-4o (Chen et al., 4 Jun 2025).
Unsafe exception handling forms another important neighboring domain. In Android development, the ExAssist tool operationalizes two techniques: XRank, which recommends likely exception types for a code snippet, and XHand, which recommends handling code in the catch block. The underlying empirical study covers 10 open-source Android apps, 66,944 commits, and 246 exception-related bugs/fixes. About 80% of these bugs caused app crashes, about 13% caused unstable behavior or partially non-functional features, and roughly 16% of the fixes swallowed the exception without taking action. ExAssist is trained from 4000 top free Android apps, with 13,969,235 classes, 16,489,415 methods, and 341,912,624 bytecode instructions (Nguyen et al., 2019).
The predictive performance is high. XRank reaches top-1 accuracy of 70% and top-3 accuracy of 87% for exception-type recommendation. XHand reaches 89% and 96%, respectively, for handling-code recommendation. Using a 25% confidence threshold, ExAssist recommends adding exception handling for 116 of 128 exception bugs in an evaluation dataset, with the correct exception type appearing in the top-1 recommendation for 86 cases, top-2 for 96, and top-3 for 104. The work therefore treats “bad code” not only as incorrect logic but also as missing handlers, swallowed exceptions, overly broad catches, and non-repairing handlers (Nguyen et al., 2019).
Language-specific bad-code catalogs provide a third adjacent strand. A grey literature review on Elixir reports that developers discuss 11 of Fowler and Beck’s 22 traditional smells and proposes 18 new smells specific to Elixir systems. The observed traditional smells include Comments, Long Parameter List, Feature Envy, Shotgun Surgery, Duplicated Code, Long Function, Large Class, Inappropriate Intimacy, Divergent Change, Speculative Generalization, and Primitive Obsession. The Elixir-specific smells include GenServer Envy, Agent Obsession, Unsupervised process, Large messages between processes, Complex multi-clause function, Exceptions for control-flow, Compile-time app configuration, and others tied to OTP, BEAM, macros, and configuration practices. The study concludes that Credo automatically detects only two traditional smells and one Elixir-specific smell (Vegi et al., 2022).
Taken together, these works expand EliBadCode beyond a single mechanism. They show that “bad code” can be dead, exception-prone, architecture-specific, or maintenance-hostile, and that high-performing systems increasingly combine classification, localization, explanation, and repair rather than simple warning generation.
5. Repository-scale semantic analysis of bad code
A further adjacent development is the use of Code Property Graphs with LLMs for security analysis at repository scale. The codebadger system integrates Joern with LLMs through an MCP server, exposing high-level tools such as program slicing, taint tracking, data flow analysis, semantic code navigation, get_call_graph, get_program_slice, get_data_dependencies, find_taint_flows, and find_bounds_checks (Lekssays, 25 Mar 2026).
The motivating claim is that plain LLMs face three critical limitations in vulnerability analysis of real-world repositories: token limits, failure to capture inter-procedural data flows, and difficulty generating complex CPGQL queries. codebadger addresses this by letting the graph engine do the heavy semantic work while the LLM orchestrates the analysis. The underlying CPG unifies AST, CFG, and PDG structure, and the paper presents explicit algorithms for taint flow analysis and backward program slicing over a graph (Lekssays, 25 Mar 2026).
The system is demonstrated on three use cases. First, it supports navigation of an 8,000-method codebase; in the reported GGML case the summary exposed 8,667 methods and 198,006 function calls, after which the LLM located 54 allocation sites and 300+ dangerous operations and identified unbounded alloca, integer overflows in allocation-size arithmetic, unchecked pointer arithmetic, and related memory-safety patterns. Second, it helped discover and exploit a previously unreported buffer overflow in libtiff, with AddressSanitizer confirming a heap-buffer-overflow. Third, it generated a correct patch for an integer overflow vulnerability (CVE-2025-6021) in libxml2 on the first attempt, closely matching the maintainers’ final fix (Lekssays, 25 Mar 2026).
This work is not itself named EliBadCode, but it is directly relevant to the broader interpretation of eliminating bad code. It shifts the focus from single files or generated snippets to semantically grounded, inter-procedural reasoning over entire repositories.
6. Significance, limitations, and open directions
The EliBadCode literature, in its multiple senses, converges on a common methodological point: problematic code is rarely reducible to shallow syntax. In the LLM-generated-code study, errors include semantic cryptographic failure, buffer-size misreasoning, missing defensive checks, and prompt-induced regressions (Chong et al., 2024). In the backdoor-defense work, the problem lies inside the model, where poisoned behavior is latent until a trigger is present (Sun et al., 2024). In DCE-LLM, bad code includes semantically unreachable or unused regions that can mislead both humans and models (Chen et al., 4 Jun 2025). In ExAssist and the Elixir smell review, the emphasis shifts to exception anti-patterns and language-specific architecture misuse (Nguyen et al., 2019, Vegi et al., 2022). In codebadger, the challenge is semantic analysis at repository scale rather than at snippet scale (Lekssays, 25 Mar 2026).
The limitations are correspondingly heterogeneous. The AI-generated-code study is restricted to C and to GPT-4o, even though its task suite is broad (Chong et al., 2024). EliBadCode for neural code models assumes white-box access, focuses on identifier triggers, and needs around 30 clean samples per class for reliable trigger inversion (Sun et al., 2024). DCE-LLM is constrained by CodeBERT input length of 512 tokens and does not claim to detect all dead code (Chen et al., 4 Jun 2025). ExAssist is specialized to Java/Android and to runtime-exception patterns learned from mobile-app corpora (Nguyen et al., 2019). The Elixir review is based on grey literature and examines only the top 60 Google results from its search procedure (Vegi et al., 2022). codebadger inherits the computational overhead of Joern and the static-analysis limits of CPG-based reasoning (Lekssays, 25 Mar 2026).
A plausible synthesis is that future EliBadCode-style systems will combine several presently separate ideas: rigorous static and dynamic validation of generated code, model-level defenses against poisoned code intelligence, attribution-guided localization of problematic regions, structured explanation and patch generation, and graph-based semantic tooling for large repositories. The literature already points in that direction. The secure-code-generation study explicitly recommends unit testing, fuzzing, sanitizers, static analysis, and manual review for LLM-produced code (Chong et al., 2024). DCE-LLM shows that specialized fine-tuning plus attribution substantially outperforms generic prompting for dead-code tasks (Chen et al., 4 Jun 2025). codebadger demonstrates that repository-scale program analysis becomes more tractable when high-level semantic tools mediate between the LLM and the code graph (Lekssays, 25 Mar 2026).
In contemporary research usage, therefore, EliBadCode names both a warning and a program. It warns against blind trust in AI-mediated coding, and it names a broader scientific agenda: identifying bad code precisely, understanding why it is bad, and removing or neutralizing it without sacrificing the intended functionality of code or the normal utility of the models that reason about code.
