Security Regression: Concepts Overview

• Security regression is the study of how system updates can degrade previously secure states by introducing vulnerabilities in software, ML models, and statistical frameworks.
• It encompasses methodologies such as regression testing, longitudinal security analysis, and continual learning adjustments that quantify changes in security metrics.
• Practical insights include the need for improved tooling, targeted security testing in maintenance phases, and privacy-preserving regression methods in adversarial environments.

Searching arXiv for recent and foundational papers on security regression and adjacent usages of the term. arxiv_search(query="security regression software vulnerabilities regression testing continual learning malware detection", max_results=10, sort_by="relevance") Security regression denotes several closely related change-sensitive concepts in the security literature. In software engineering, it most commonly refers to vulnerabilities introduced into a previously unaffected software system by source-code changes, often during bug fixing (Braz et al., 2022). Closely related work defines security regression testing as testing that ensures that changes applied to a system do not harm security and is applied during maintenance (Pekaric et al., 2023). In continual-learning security systems, the term has been extended to harmful prediction reversals after model updates, such as malware samples that were previously detected but later evade detection (Ghiani et al., 24 Jul 2025). Empirical software-engineering work also uses a security-regression framing to study whether security issue density improves or worsens over time under specific process interventions (Rass et al., 13 May 2026). This suggests that the unifying idea is not a static notion of insecurity, but deterioration relative to a prior secure or safer state.

1. Terminological scope

The literature uses “security regression” in several non-identical but structurally similar ways. Some usages are about software evolution, some about ML model updates, and some about statistical security outcomes tracked over time. Adjacent work further applies regression methods to security-sensitive prediction tasks or to secure computation itself, which broadens the terminological field without making the underlying concepts identical (Zhang et al., 27 Nov 2025, Meng et al., 2019, Chiang, 2022, Charalambides et al., 2022).

Usage	Core object	Representative definition or metric
Software security regression	Code change	Vulnerabilities introduced in a previously unaffected system
Security regression testing	Maintenance testing	Ensures changes do not harm security
ML security regression	Model update	Negative flips and negative flip rate
Longitudinal security regression analysis	Team or project trajectory	Security issue density $Q_t$ and relative improvement ratio $\Delta Q$
Prompt-level security regression task	LLM prompt	Predicting ASR or relative ASR ranking

In the software-engineering sense, the regression is the introduction of a new weakness after a change. In the continual-learning sense, the regression is a harmful change in prediction behavior at the sample level. In the longitudinal process-analytics sense, regression is measured by whether scanner-detected security issues accumulate relative to code growth. A plausible implication is that the term functions as a family resemblance concept centered on post-change security degradation rather than on a single formalism.

2. Regression vulnerabilities in software evolution

An exploratory mixed-method case study of Mozilla provides a concrete operationalization of security regressions as regression vulnerabilities introduced by bug-fixing activity. The study mined Mozilla Bugzilla using the public security keywords sec-high, sec-critical, sec-moderate, sec-incident, and sec-low, together with a non-empty Regressed by field. From 6,667 public bug reports of security issues, filtering by Regressed by yielded 187 regression vulnerability reports; after exclusions, the analysis used 78 regression vulnerability reports and 72 vulnerability-inducing bug reports, plus five semi-structured interviews of 30 minutes each with Mozilla developers involved in the fixes (Braz et al., 2022).

The empirical picture is notably specific. Among the 78 regression vulnerabilities, 68 occurred in Mozilla Core, 5 in Toolkit, 3 in Firefox, 1 in NSS, and 1 in GeckoView. Their bug reports remained open for a median of 11 days. The 72 vulnerability-inducing bug reports had a median closure time of 37.5 days and spanned February 2007 to December 2020. The inducing defects were classified into six categories, dominated by structural defects at 34 cases and resource defects at 15 cases; the paper emphasizes that structural and resource defects are prevalent causes of bugs, whereas most of the regression vulnerabilities induced by the fixes of these bugs occur due to defects related to resources. The 72 inducing bug fixes contained 116 flaws classified into 26 CWEs, with CWE-416 “Use after free” occurring 34 times, followed by CWE-703 at 16 and several other memory- and resource-lifetime-related weaknesses at lower frequencies (Braz et al., 2022).

The behavioral findings are equally central to the concept. Security was rarely discussed during bug fixes: only 5 out of 72 bug reports contained comments expressing concern about the security impact of the fix, and the study summarizes this as 6% of analyzed bug reports. Developers’ main concerns were the complexity of the bug and community pressure to fix it. The interviews further report that Mozilla had no guideline specifically to avoid regression vulnerabilities during bug fixes. Detection relied heavily on tooling but not exclusively: 30 regression vulnerabilities were found by Mozilla users, 24 by dynamic analysis tools such as Address Sanitizer, 15 through crashes due to user usage, and 10 through assertion failures. The study concludes that dynamic analysis tools helped finding around 30% of regression vulnerabilities, but that tool support alone was not enough to ensure security during bug fixes (Braz et al., 2022).

These findings make the notion of security regression sharply empirical. The regression is not merely an abstract possibility after change; it is a recurrent outcome associated with structural edits, resource handling, incomplete security discussion, and partial reliance on post hoc detection.

3. Security regression testing and the maintenance problem

Security regression testing is defined in automotive security-testing literature as follows: “Regression testing ensures that changes applied to the system do not harm the security and is applied during maintenance.” In a systematic mapping of 39 selected papers on automotive security testing, regression testing is one of five security testing technique categories, alongside model-based testing, code-based testing and static analysis, penetration testing and dynamic analysis, and risk-based testing (Pekaric et al., 2023).

The mapping study’s central result is negative but consequential: it found no automotive security testing approaches focused on regression testing. This absence recurs across the study’s classification scheme. In the lifecycle analysis, the authors state that they did not encounter any approaches focusing on regression testing. In the AUTOSAR-layer analysis, “Regression Testing” is zero across all layers. In the functional-interface analysis, regression testing is again zero across all interfaces. In the attack-analysis table, regression testing has zero for all attack categories. The same study reports no approaches for maintenance and service or for decommissioning, even though those are the lifecycle phases in which regression testing would naturally operate (Pekaric et al., 2023).

The classification framework clarifies why this gap matters. The automotive paper organizes the literature by security testing techniques, AUTOSAR layers, AUTOSAR functional interfaces, vehicle lifecycle phases derived from ISO/SAE 21434, and attack types. It also points to a broader security-regression-testing taxonomy with dimensions such as abstraction level, security issue, regression testing techniques, tool support, evaluated system, maturity of the system, and evaluation measures. Yet those regression-specific dimensions could not be populated for automotive engineering because the mapped corpus contained no regression-testing approaches in scope (Pekaric et al., 2023).

The significance is twofold. First, the absence is itself a research result: modern vehicles change over time, but the surveyed automotive security-testing literature remains concentrated in design and engineering, production, and operation by customer. Second, the paper explicitly links this absence to security needs in maintenance and service, arguing that security measures installed during design may be disrupted throughout the operational phase and that changing security requirements require updates. In that sense, security regression testing is less a mature automotive subfield than an identified deficiency in lifecycle coverage.

4. Longitudinal measurement of security degradation and improvement

A distinct but related usage treats security regression as a longitudinal outcome measurable sprint by sprint. A controlled course experiment with 84 students across 14 teams and 3 sprints asked whether incentivizing security improvements through bonus points leads to lower security issue density than equivalent incentives for general code quality. The design compared a control condition, incentivized for SonarQube code quality results, with a treatment condition, incentivized for reducing security issues. The key feature was that the same course, grading structure, maximum bonus, available scan reports, and sprint structure were held constant; only the reward target differed (Rass et al., 13 May 2026).

The paper formalizes three metrics. Let $I_t$ be the total number of security issues found at time $t$. Security issue density is

$Q_t=\frac{I_t}{\text{LOC}_t}.$

The relative improvement ratio across consecutive sprint measurements is

$$\Delta Q=\frac{Q_{t_{i+1}}}{Q_{t_i}}, \qquad \Delta Q\in[0,\infty).$$

The interpretation is explicit: $\Delta Q<1$ indicates improvement, $\Delta Q=1$ no change, and $\Delta Q>1$ worsening. This turns security regression into a ratio-based trajectory question: whether issue density improves or worsens relative to code volume growth (Rass et al., 13 May 2026).

The study used a semi-automated measurement pipeline. Bearer, Detekt, and mobsfscan were run on the main branch at each assessment point; reports were produced in SARIF format; Jenkins was used for CI/CD; reports were emailed to students and logged in a database; and LOC were computed using a Python script that counted .java and .kt files while excluding comments. The main statistical model was beta regression with a logit link, using security issue density as the dependent variable. The global comparison reported $\text{Estimate}(grp=SEC)=-0.3963$ with $\Delta Q$0, which the paper interprets as significantly lower security issue density in the security-incentivized condition. Poisson regression on LOC indicated that code volume increased similarly across groups, so the result was not attributed to reduced output. After controlling for platform, the paper reports a marked front-end/back-end disparity: back-ends showed fewer issues and higher improvement ratios under incentives, while front-ends accumulated more issues; the platform effect dominated in the more detailed density model, and a regression on $\Delta Q$1 did not find statistically significant differences between groups or layers (Rass et al., 13 May 2026).

This strand of work does not define security regression as a single vulnerability introduced by one patch. Instead, it operationalizes regression as a temporal process in which security issue density may deteriorate or improve. The shared conceptual core remains change over time, but the unit of analysis shifts from a code change to a sprint-level development trajectory.

5. Sample-level security regression in continual malware detection

In continual learning for Android malware detection, security regression is formalized as harmful prediction inconsistency after model updates. The paper distinguishes it from catastrophic forgetting. A negative flip occurs when the old model is correct and the new model is wrong: $\Delta Q$2 The negative flip rate is

$\Delta Q$3

For continual learning over experiences $\Delta Q$4, the paper defines a CL-specific $\Delta Q$5 between consecutive models and contrasts it with forgetting: $\Delta Q$6 together with the identity

$\Delta Q$7

The conceptual distinction is explicit: forgetting is a global average performance drop, whereas security regression captures only the harmful bad flips (Ghiani et al., 24 Jul 2025).

The empirical setting spans DIL and CIL scenarios on ELSA, TESSERACT, and AZ-Class. The classifier is an MLP with a 512-unit hidden layer, SGD, learning rate $\Delta Q$8, momentum 0.9, 30 epochs, and batch size 32. Baselines include Naive and Cumulative, with continual-learning methods Replay, A-GEM, EWC, SI, and LwF. To mitigate regression, the paper adapts Positive Congruent Training to continual learning via a regression-aware penalty that preserves prior predictive behavior in a model-agnostic manner (Ghiani et al., 24 Jul 2025).

The results show that strong aggregate performance does not preclude substantial regression. In DIL backward mode on ELSA, Naive had 3.17% malware NFR and 0.48% goodware NFR, SI had 3.40% malware NFR, and A-GEM had 3.46% malware NFR. Adding the PCT-based penalty sharply reduced regression: SI 3.40% to SI+PCT 1.19%, and A-GEM 3.46% to A-GEM+PCT 1.76%. In forward mode, where concept drift is more severe, EWC and SI on ELSA had forward NFR around 3.12% and 3.06%, whereas EWC+PCT and SI+PCT reduced them to 0.98% and 0.90%. In CIL on AZ-Class, PCT improved all CL methods; for example, SI+Replay improved from 72.46% to 78.12% accuracy, reduced forgetting from 29.99% to 22.16%, and reduced NFR from 9.76% to 7.42% (Ghiani et al., 24 Jul 2025).

This formulation is important because it makes explicit that a security-critical model update can be unacceptable even when top-line metrics look stable or improved. The regression is defined at the sample level, not at the level of mean accuracy.

6. Regression models as security objects and security predictors

Another branch of the literature studies security in regression settings themselves. In EEG-based BCI regression, white-box target adversarial attacks are defined for models with numeric outputs rather than class labels. Given original input $\Delta Q$9, true output $I_t$0, and regression model $I_t$1, the attacker seeks $I_t$2 such that

$I_t$3

while minimizing perturbation size. The paper proposes CW-R and IFGSM-R, regression adaptations of Carlini-Wagner and iterative FGSM, and evaluates them on driving/drowsiness estimation and PVT reaction time estimation using ridge regression and a two-hidden-layer MLP. With target shift $I_t$4, both attacks achieved very high ASR, generally close to 100%; CW-R had slightly higher ASR and lower distortion but much higher computational cost; Gaussian noise with similar distortion had ASR 0%; and adversarial examples transferred between MLP and RR, enabling black-box attacks. The authors state that this is the first study on adversarial attacks for EEG-based BCI regression problems and explicitly note that no defense is proposed (Meng et al., 2019).

Large-language-model security work introduces yet another usage: prompt-level security treated as a continuous regression target. A black-box jailbreak study models a prompt’s attack success rate as

$I_t$5

and trains a lightweight proxy to predict relative ASR via pairwise ranking regression rather than direct regression. The framework uses an improved Outline Filling Attack to generate dense samples near the target model’s security boundary and trains a Llama-3-8B-Instruct proxy with binary cross-entropy on pairwise labels. On 80 dangerous questions from JailbreakBench and HarmBench, with 75 outline-filled prompts per question and 20 queries per prompt, the proxy achieved 91.1% ranking accuracy for ALR and 69.2% for ASR; when used for attack prioritization, IASR increased and FASC dropped sharply on several targets (Zhang et al., 27 Nov 2025).

These studies do not define security regression in the software-maintenance sense. Rather, they show that regression models, continuous security scores, and regression-style objectives are themselves central objects in adversarial ML and LLM red teaming. A plausible implication is that security regression now names both a failure mode after change and a modeling regime in which security is quantified on a continuous scale.

7. Secure regression computation and open research directions

A separate terminological strand concerns secure or privacy-preserving regression computation. Privacy-preserving logistic regression training over encrypted data introduces a quadratic gradient $I_t$6 derived from a diagonal Hessian-based scaling matrix, and applies it to enhanced NAG and Adagrad. In homomorphic logistic regression, the enhanced NAG method reduced the encrypted training protocol from 7 iterations to 3 iterations, with faster wall-clock learning time but dataset-dependent reductions in accuracy or AUC, and with constraints from polynomial sigmoid approximation, modulus budget, and encrypted refresh cost (Chiang, 2022). Distributed secure coded regression applies random orthonormal sketches before block subsampling so that data are both protected and compressed; the 2022 and 2023 papers frame the orthonormal transform as an encoded encryption in an approximate gradient-coding scheme, interpret block subsampling as the responses of non-straggling workers, and distinguish information-theoretic secrecy for generic orthonormal projections from computational security for garbled block-SRHT constructions (Charalambides et al., 2022, Charalambides et al., 2023). In vertically partitioned sparse logistic regression, CAESAR combines homomorphic encryption and secret sharing, uses a cubic Minimax approximation to the sigmoid, and reports about 130 times speedup over SecureML on a real risk-control task while matching its accuracy metrics (Chen et al., 2020).

Across the broader field, the open problems are unusually consistent. The Mozilla study calls for further work on security tooling support and on integrating it during bug fixes, while also pointing to security-oriented development culture, stronger collaboration between developers, security experts, and testing teams, and training and guidelines for secure bug fixing (Braz et al., 2022). The automotive mapping identifies security regression testing, maintenance-phase security, decommissioning, and security-safety integration as major open research gaps (Pekaric et al., 2023). The incentive study argues that its results merit follow-up in professional contexts and longer development lifecycles (Rass et al., 13 May 2026). The continual-learning malware paper notes the clean-label assumption, the need for access to prior model outputs, and the lack of temporal structure in AZ-Class, and suggests active learning and temporally aware benchmarks as future work (Ghiani et al., 24 Jul 2025). The EEG adversarial-regression paper explicitly leaves defenses for future study (Meng et al., 2019).

Taken together, these directions show that security regression is not a narrow niche. It is a recurring analytical lens for studying how security degrades after change, how such degradation can be detected or quantified, and how secure computation can make regression models usable in adversarial or privacy-constrained environments. The literature is methodologically heterogeneous, but it converges on a common premise: security claims that ignore update dynamics, maintenance, or longitudinal drift are incomplete.