---
title: 'EliBadCode: Understanding Bad Code in AI Systems'
url: https://www.emergentmind.com/topics/elibadcode
type: topic
---

# EliBadCode: Understanding Bad Code in AI Systems

Searching arXiv for the specified papers and topic to ground the article with current bibliographic context.
EliBadCode is a research term that appears in recent software-engineering and code-intelligence literature in more than one sense. In one usage, it is a shorthand for the danger of naively trusting AI-generated code, especially when code produced by large language models is deployed without rigorous verification. In another, it is the name of a backdoor-elimination framework for neural code models, designed to reverse-engineer identifier-based triggers and unlearn malicious behavior. A broader interpretation, explicitly suggested in later work on dead-code elimination, treats “EliBadCode” as the general task of detecting, explaining, and correcting problematic code, including dead code, unsafe exception handling, language-specific smells, and semantically subtle vulnerabilities across large repositories [2409.19182] [2408.04683] [2506.11076].

## 1. Terminological scope and conceptual frame

Across the cited works, EliBadCode does not denote a single universally fixed artifact. The term is used explicitly as a shorthand in a study of AI-generated C code, where it names the danger of blindly trusting model outputs, and as a formal framework in work on securing neural code models against backdoor attacks [2409.19182] [2408.04683]. A broader interpretive extension appears in work on dead-code elimination, where the query “EliBadCode” is treated as the idea of a system that can eliminate bad code by locating, explaining, and repairing problematic patterns [2506.11076].

This dual use is significant because it spans two layers of the software stack. One layer concerns code artifacts themselves: incorrect implementations, insecure memory behavior, dead branches, swallowed exceptions, and code smells. The other concerns the models and tools that analyze or generate code: code LLMs that emit brittle implementations, or neural code models that can be backdoored. Taken together, the literature suggests that “bad code” is not merely stylistic deficiency. It includes semantic incorrectness, exploitable memory behavior, omitted defensive programming, poisoned model behavior, and structurally misleading code that interferes with downstream analysis.

A plausible implication is that EliBadCode is best understood not as one method but as a family of research problems at the intersection of secure code generation, program analysis, automated repair, and trustworthy code intelligence.

## 2. AI-generated code considered harmful

In the usage introduced by the secure-code-generation study, EliBadCode denotes the risk of treating LLM-generated code as trustworthy by default. That study evaluates **C** code generated by **OpenAI GPT-4o**, using an evaluation framework called **EXACT** (**EXamination System for AI-Generated Code Testing**) across three task classes: **202 LeetCode-style problems**, **data structures and algorithms** drawn from repositories including “The Algorithms,” “fragglet,” and “petewarden,” and **cryptographic routines** including **SHA-1, MD5, and AES** from OpenBSD [2409.19182].

EXACT combines **code generation**, **functionality validation**, **security analysis**, and **complexity/size measurement**. Functionality validation uses the **LeetCode online judge** for interview-style tasks, **CUnit** and **AFL** for DS&A code, and **NSRL** crypto test vectors for cryptographic routines. Security analysis uses the **Clang static analyzer** with default plus “experimental” core, security, and Unix checkers. Complexity and size are measured using **SCC**, including **cyclomatic complexity**, **lines of code**, and **complexity per line** [2409.19182].

The central empirical result is that GPT-4o-generated code is frequently functional enough to appear useful, yet systematically less robust and less secure than human-written code. On the **202 LeetCode problems**, **87.6%** of GPT-4o solutions passed, while **12.4%** failed. Among the failures, **9** were due to failing test cases or **Time Limit Exceeded**, **12** were runtime errors with sanitizers enabled, and **4** were compile errors caused by wrong function signatures. The runtime errors included **heap buffer overflow**, **stack buffer overflow**, **array index out of bounds**, **signed integer overflow**, **load of address with insufficient space**, and **requested allocation size exceeds maximum supported size**. Crucially, **8 of these 12 runtime errors were not detected by Clang static analysis**, indicating that static analysis alone was insufficient to surface serious faults [2409.19182].

The paper reports that difficulty matters. Of the **21** LeetCode tasks GPT-4o failed, **20** were labeled **medium or hard**. In the DS&A setting, GPT-4o versions of **15** tasks passed unit tests, whereas the human code had **3/15** unit-test failures; however, fuzzing reversed the picture of reliability. GPT-4o code produced **12 unique hangs** versus **6** for human code, and **451 unique crashes** versus **364**, yielding **50% more hangs** and **23.9% more crashes**. The conclusion was not that human code was flawless, but that LLM code was systematically more fragile under adversarial or unexpected inputs [2409.19182].

The cryptographic results are especially consequential. GPT-4o implementations of **MD5** and **AES** produced correct outputs on all test vectors, but the **SHA-1** implementation produced **incorrect hash values for all inputs**, despite compiling and running. The error was therefore silent and semantic rather than syntactic. In security-critical code, that distinction is decisive: a routine may appear structurally valid while violating the intended cryptographic specification [2409.19182].

The study also isolates characteristic failure modes. A toy buffer-size task shows that GPT-4o often reasons by pattern rather than by actual memory-safety requirements: in an alphabet string missing the letter **G**, the correct size is **25**, yet the model often filled **26**, causing heap overflow. Across **1000 trials per definition**, success rates ranged from **99.3%** for a constant integer, to **41.6%** for the missing-letter alphabet case, to **1.5%** for float × float, with rare outputs of gibberish non-English characters instead of numbers. This suggests unreliable numeric reasoning in contexts where exact arithmetic is required for memory safety [2409.19182].

The security delta also appears in static analysis. Over **220 files** (**202 LeetCode + 18 DS&A**), GPT-4o code incurred **128** Clang issues versus **116** for human code, or **10.3% more static-analysis issues** overall. The largest disparity was **malloc overflow**, with **81** issues for GPT-4o and **61** for human code, or **32.8% more** in LLM-generated code. Meanwhile, GPT-4o code was shorter but denser: mean **LoC** was **23.6** versus **29.6** for LeetCode and **103.5** versus **192.1** for DS&A, yet **complexity per line** was higher for GPT-4o in both settings, **0.33** versus **0.28** for LeetCode and **0.21** versus **0.17** for DS&A. The paper characterizes this output as “bare-bones”: compressed logic with fewer explicit checks, fewer guardrails, and more brittle semantics [2409.19182].

Finally, the study shows that prompt-based repair is unreliable. A feedback loop over **60 GPT-4o LeetCode solutions** reduced some vulnerabilities but introduced others. In files that already had issues, total targeted issues dropped from **58** to **44** after one regeneration loop, but **array index out of bounds** increased from **15** to **18**. In **30** files initially free of the targeted Clang issues, regeneration introduced **4 new malloc overflow issues**. Ancillary prompting experiments were also non-monotonic: asking for **shorter line lengths** yielded **11.3% lower** cyclomatic complexity per line and fixed **17.9%** of Clang issues, whereas asking for **fewer lines of code** made code **18.4% more complex per line** but fixed **43.2%** of Clang issues. No simple prompt recipe emerged as a reliable security control [2409.19182].

## 3. EliBadCode as backdoor elimination for neural code models

A distinct meaning of EliBadCode appears in work on **neural code models** used for **defect detection**, **clone detection**, and **code search**. Here EliBadCode is a defense framework for eliminating backdoors implemented through **identifier renaming** triggers. The threat model assumes a defender with **white-box access** to the model, access to a **small clean dataset**, and no access to the original poisoned training set or true trigger. The attacker poisons training data so that the model behaves normally on clean code but flips to an attacker-chosen output when code contains a stealthy identifier trigger [2408.04683].

The framework is organized around **trigger inversion** followed by **unlearning**. It first filters the model vocabulary according to programming-language naming rules, reducing the trigger search space to tokens that plausibly occur in identifiers. For **CodeBERT**, this reduces the vocabulary from **50,265** tokens to **15,838** trigger candidates. It then identifies **sample-specific trigger positions** by masking identifier positions and measuring the loss difference induced by masking. The least sensitive position is selected for each sample, on the empirical premise that true backdoor triggers are more position-robust than ordinary adversarial perturbations [2408.04683].

Trigger inversion is then performed with a **Greedy Coordinate Gradient** procedure over discrete token sequences. For a candidate target label \(y'\), the framework minimizes an inversion loss of the form
\[
\mathcal{L}_{\mathrm{inv}}(t, y', \theta^*)
=
\mathbb{E}_{s^m \in S^m}
\left[
\mathcal{L}\big(f(s^m \oplus t; \theta^*), y'\big)
\right],
\]
where \(t\) is the trigger, \(S^m\) is a masked clean subset, and \(\theta^*\) are the backdoored parameters. The label for which inversion yields the lowest loss is treated as the backdoor target. EliBadCode then applies **trigger anchoring**, removing tokens whose deletion causes only small loss change; with threshold **\(\beta = 0.15\)**, effective trigger tokens are separated from noise tokens [2408.04683].

Backdoor elimination is performed by **model unlearning**. The anchored trigger is injected into a fraction of clean samples, and only the **last layer parameters** are fine-tuned, rather than the entire model, to avoid catastrophic forgetting. The paper reports that a trigger injection rate of roughly **20%** is sufficient to suppress attack behavior while preserving normal performance [2408.04683].

Evaluation covers **CodeBERT**, **CodeT5**, and **UniXcoder** on **Devign** for defect detection, **BigCloneSearch** for clone detection, and **CSN-Python** for code search. Two attacks are considered: **CodePoisoner**, using trigger **“testo_init”**, and **BadCode**, using trigger **“rb”**. Under **CodePoisoner**, average **ASR** drops from **99.04%** to **0.93%** for defect detection and from **100%** to **4.73%** for clone detection, while code-search **ANR** rises from **9.55** to **24.95**; clean **ACC** on defect detection changes from **64.14%** to **63.40%**, and clone-detection **F1** changes from **94.15%** to **96.70%**. Under **BadCode**, average **ASR** drops from **99.76%** to **2.64%** for defect detection and from **100%** to **6.11%** for clone detection, while code-search **ANR** rises from **9.99** to **25.18**. A baseline defense, **DBS**, is markedly less effective in these settings [2408.04683].

The ablation results clarify which components are essential. Without **PL-specific vocabulary filtering**, trigger inversion fails and post-defense **ASR** remains **100%**. Removing **sample-specific position identification** leaves end performance largely unchanged but increases inversion cost from roughly **25 epochs** to **52 epochs**. Removing **trigger anchoring** can push **ASR** even lower, but at a notable cost in clean accuracy: in one CodeBERT defect-detection setting, **ACC** falls to **60.98%** instead of **62.57%** with anchoring. The framework also requires enough clean data: with fewer than **20 clean samples**, inversion fails, whereas **30 samples per class** gave the best reported performance [2408.04683].

In this sense, EliBadCode is not about generated source code directly. It is a model hardening technique for securing code-understanding systems themselves.

## 4. Related efforts to detect, explain, and repair problematic code

A broader “eliminate bad code” interpretation is made explicit in work on **dead code elimination**. **DCE-LLM** defines dead code as **unused** or **unreachable** code and frames it as both a software-engineering and a security problem, including its role in **malware & obfuscation** and in **LLM vulnerabilities** such as dead-code injection attacks. The system combines a **CodeBERT-based pivot model** with attribution-based line selection and a fine-tuned **Qwen2-7B-Instruct** expert model that outputs a structured judgment, line numbers, type labels, explanations, and fixed code. It is trained on **AIDCE**, a dataset of **19,639** samples from CodeNet with **12,628 normal**, **4,495 unused**, and **2,516 unreachable** samples, plus an SFT dataset of approximately **1,500** samples using GPT-4o-generated silver explanations and fixes [2506.11076].

Its performance is substantially above generic LLM baselines. On the AIDCE test split, **DCE-LLM** achieves **F1 = 94.02** for **unused**, **96.65** for **unreachable**, and **99.38** for **normal**, with **overall accuracy = 96.40**. By contrast, **GPT-4o (zero-shot)** achieves **F1 = 40.98** for unused, **64.00** for unreachable, **76.96** for normal, and **accuracy = 60.97**. The paper summarizes this as **over 94% F1** on unused and unreachable code and a performance margin of more than **30% F1** over GPT-4o on dead-code classes. In an unseen **Golang** setting, DCE-LLM still reaches **F1 = 90.42** for unused and **90.91** for unreachable, compared with **67.59** and **81.87** for GPT-4o [2506.11076].

Unsafe exception handling forms another important neighboring domain. In Android development, the **ExAssist** tool operationalizes two techniques: **XRank**, which recommends likely exception types for a code snippet, and **XHand**, which recommends handling code in the catch block. The underlying empirical study covers **10 open-source Android apps**, **66,944 commits**, and **246 exception-related bugs/fixes**. About **80%** of these bugs caused app crashes, about **13%** caused unstable behavior or partially non-functional features, and roughly **16%** of the fixes swallowed the exception without taking action. ExAssist is trained from **4000 top free Android apps**, with **13,969,235 classes**, **16,489,415 methods**, and **341,912,624 bytecode instructions** [1908.06567].

The predictive performance is high. **XRank** reaches **top-1 accuracy of 70%** and **top-3 accuracy of 87%** for exception-type recommendation. **XHand** reaches **89%** and **96%**, respectively, for handling-code recommendation. Using a **25%** confidence threshold, ExAssist recommends adding exception handling for **116 of 128** exception bugs in an evaluation dataset, with the correct exception type appearing in the **top-1** recommendation for **86** cases, **top-2** for **96**, and **top-3** for **104**. The work therefore treats “bad code” not only as incorrect logic but also as missing handlers, swallowed exceptions, overly broad catches, and non-repairing handlers [1908.06567].

Language-specific bad-code catalogs provide a third adjacent strand. A grey literature review on Elixir reports that developers discuss **11** of Fowler and Beck’s **22** traditional smells and proposes **18 new smells** specific to Elixir systems. The observed traditional smells include **Comments**, **Long Parameter List**, **Feature Envy**, **Shotgun Surgery**, **Duplicated Code**, **Long Function**, **Large Class**, **Inappropriate Intimacy**, **Divergent Change**, **Speculative Generalization**, and **Primitive Obsession**. The Elixir-specific smells include **GenServer Envy**, **Agent Obsession**, **Unsupervised process**, **Large messages between processes**, **Complex multi-clause function**, **Exceptions for control-flow**, **Compile-time app configuration**, and others tied to OTP, BEAM, macros, and configuration practices. The study concludes that **Credo** automatically detects only **two traditional smells** and **one Elixir-specific smell** [2203.08877].

Taken together, these works expand EliBadCode beyond a single mechanism. They show that “bad code” can be dead, exception-prone, architecture-specific, or maintenance-hostile, and that high-performing systems increasingly combine classification, localization, explanation, and repair rather than simple warning generation.

## 5. Repository-scale semantic analysis of bad code

A further adjacent development is the use of **Code Property Graphs** with language models for security analysis at repository scale. The **codebadger** system integrates **Joern** with LLMs through an **MCP server**, exposing high-level tools such as **program slicing**, **taint tracking**, **data flow analysis**, **semantic code navigation**, **get_call_graph**, **get_program_slice**, **get_data_dependencies**, **find_taint_flows**, and **find_bounds_checks** [2603.24837].

The motivating claim is that plain LLMs face three critical limitations in vulnerability analysis of real-world repositories: **token limits**, failure to capture **inter-procedural data flows**, and difficulty generating complex **CPGQL** queries. codebadger addresses this by letting the graph engine do the heavy semantic work while the LLM orchestrates the analysis. The underlying CPG unifies **AST**, **CFG**, and **PDG** structure, and the paper presents explicit algorithms for **taint flow analysis** and **backward program slicing** over a graph \(G\) [2603.24837].

The system is demonstrated on three use cases. First, it supports navigation of an **8,000-method** codebase; in the reported **GGML** case the summary exposed **8,667 methods** and **198,006 function calls**, after which the LLM located **54** allocation sites and **300+** dangerous operations and identified unbounded `alloca`, integer overflows in allocation-size arithmetic, unchecked pointer arithmetic, and related memory-safety patterns. Second, it helped discover and exploit a previously unreported **buffer overflow in libtiff**, with **AddressSanitizer** confirming a **heap-buffer-overflow**. Third, it generated a correct patch for an **integer overflow vulnerability (CVE-2025-6021) in libxml2 on the first attempt**, closely matching the maintainers’ final fix [2603.24837].

This work is not itself named EliBadCode, but it is directly relevant to the broader interpretation of eliminating bad code. It shifts the focus from single files or generated snippets to semantically grounded, inter-procedural reasoning over entire repositories.

## 6. Significance, limitations, and open directions

The EliBadCode literature, in its multiple senses, converges on a common methodological point: problematic code is rarely reducible to shallow syntax. In the LLM-generated-code study, errors include semantic cryptographic failure, buffer-size misreasoning, missing defensive checks, and prompt-induced regressions [2409.19182]. In the backdoor-defense work, the problem lies inside the model, where poisoned behavior is latent until a trigger is present [2408.04683]. In DCE-LLM, bad code includes semantically unreachable or unused regions that can mislead both humans and models [2506.11076]. In ExAssist and the Elixir smell review, the emphasis shifts to exception anti-patterns and language-specific architecture misuse [1908.06567] [2203.08877]. In codebadger, the challenge is semantic analysis at repository scale rather than at snippet scale [2603.24837].

The limitations are correspondingly heterogeneous. The AI-generated-code study is restricted to **C** and to **GPT-4o**, even though its task suite is broad [2409.19182]. EliBadCode for neural code models assumes **white-box access**, focuses on **identifier triggers**, and needs around **30 clean samples per class** for reliable trigger inversion [2408.04683]. DCE-LLM is constrained by **CodeBERT** input length of **512 tokens** and does not claim to detect all dead code [2506.11076]. ExAssist is specialized to **Java/Android** and to runtime-exception patterns learned from mobile-app corpora [1908.06567]. The Elixir review is based on **grey literature** and examines only the **top 60 Google results** from its search procedure [2203.08877]. codebadger inherits the computational overhead of **Joern** and the static-analysis limits of CPG-based reasoning [2603.24837].

A plausible synthesis is that future EliBadCode-style systems will combine several presently separate ideas: rigorous static and dynamic validation of generated code, model-level defenses against poisoned code intelligence, attribution-guided localization of problematic regions, structured explanation and patch generation, and graph-based semantic tooling for large repositories. The literature already points in that direction. The secure-code-generation study explicitly recommends unit testing, fuzzing, sanitizers, static analysis, and manual review for LLM-produced code [2409.19182]. DCE-LLM shows that specialized fine-tuning plus attribution substantially outperforms generic prompting for dead-code tasks [2506.11076]. codebadger demonstrates that repository-scale program analysis becomes more tractable when high-level semantic tools mediate between the LLM and the code graph [2603.24837].

In contemporary research usage, therefore, EliBadCode names both a warning and a program. It warns against blind trust in AI-mediated coding, and it names a broader scientific agenda: identifying bad code precisely, understanding why it is bad, and removing or neutralizing it without sacrificing the intended functionality of code or the normal utility of the models that reason about code.

Source: https://www.emergentmind.com/topics/elibadcode