Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs

Abstract: LLMs have recently achieved strong performance in code generation. However, due to knowledge cut-off and the rapid evolution of software libraries, they often generate deprecated API usages that lead to unreliable and incompatible code. Existing fine-tuning methods lack selectivity when only a small portion of model knowledge requires modification. Recent model-level approaches, such as machine unlearning and model editing, offer a promising direction for modifying parametric knowledge. However, their use for deprecated API mitigation remains largely unexplored. Moreover, existing methods primarily suppress outdated APIs, but do not explicitly steer models toward correct replacements, often leading to mismatched or incomplete generations. To address this limitation, we developed CURE, a contrastive unlearning approach that shifts unlearning from purely suppressing outdated knowledge to explicitly promoting correct API replacements. Concretely, CURE jointly discourages deprecated APIs while encouraging their valid alternatives, enabling more reliable adaptation to evolving software libraries. The experiments on recent deprecated API benchmark dataset show that CURE not only reduces deprecated API usage but also improves correct API replacement, while preserving general code generation performance. CURE substantially outperforms two SOTA baselines with respect to different quality metrics. These findings highlight the importance of combining suppression with replacement when adapting LLMs to evolving software ecosystems.

• The paper introduces CURE, a dual-objective framework that unlearns deprecated API usage while promoting valid updates.
• It employs a contrastive loss to prioritize updated APIs over obsolete ones, reducing Deprecated API Usage Rate to as low as 4.2%.
• Experimental results demonstrate that CURE maintains HumanEval performance while significantly improving alignment with evolving APIs.

Contrastive Unlearning for Knowledge Alignment in Code LLMs

Problem Overview

The proliferation of LLMs pretrained on public code corpora has advanced code generation, but their parametric knowledge frequently encodes outdated patterns, especially deprecated API usages. Such knowledge bottlenecks not only stem from static data and knowledge cutoffs but are exacerbated by the rapid evolution of software libraries, resulting in unreliable and incompatible generations. Conventional mitigation via prompt-based or output-level strategies (e.g., prompt augmentation, output substitution) suffers from limited selectivity and strong assumptions about context-API correspondence, while fine-tuning approaches are blunt instruments for local knowledge editing. Existing model-level approaches—model editing, suppression-based machine unlearning—focus on suppressing obsolete facts but rarely incentivize the adoption of approved alternatives, yielding mismatches and incomplete replacements. The paper introduces CURE, a contrastive unlearning framework designed to promote correct API transitions at the parametric level, systematically bridging the gap between knowledge deletion and knowledge alignment.

Methodology

CURE operationalizes replacement-aware unlearning through a dual-objective optimization that suppresses deprecated APIs while explicitly rewarding generation of their valid alternatives—the latter mechanism absent from classical suppression paradigms. Specifically, for each code context $x$, CURE utilizes a triplet $(x, y^-, y^+)$, where $y^-$ uses deprecated and $y^+$ uses up-to-date APIs. The approach builds upon state-of-the-art parametric unlearning losses (PROD [jiang2026large], SimNPO [fan2024simplicity]), augmenting them with a contrastive loss ensuring that, under any context, the model places higher probability on $y^+$ than $y^-$.

• Suppression Term (Unlearning Loss): Either a sequence-level suppression (SimNPO) or a token-level probabilistic redistribution (PROD).
• Replacement Promotion (Contrastive Loss): Enforces via a sigmoid-log objective that $P(y^{+}|x) > P(y^{-}|x)$, normalized by sequence length.
• Overall Objective: $$\mathcal{L}_{CURE} = \mathcal{L}_{unl} + \lambda \mathcal{L}_{ctr}$$, with a single tunable parameter $\lambda$ governing trade-off.

Underlying datasets are constructed from a benchmark mapping deprecated and updated APIs spanning 8 Python libraries. Hard negatives from up-to-date sets and Gemini-2.5-Pro rewritten positives for contrastive samples induce robustness and semantic preservation.

Experimental Evaluation

The evaluation is conducted on four representative Code LLMs (DeepSeek, StarCoder2, CodeLlama, CodeGen) across three main axes: Deprecated API Usage Rate (DUR), Replacement API Usage Rate (RUR), and Mismatch API Usage Rate (MUR). Model utility is assessed on HumanEval (Pass@k).

• Evaluation is performed on both $\mathcal{D}_f$ (forgetting/outdated contexts) and $(x, y^-, y^+)$0 (up-to-date, unseen contexts).
• DUR, RUR, and MUR quantify, respectively: persistence of deprecated APIs, correct replacement adoption, and failure to align on either.

The primary findings are visualized as follows.

Figure 1: DUR, RUR, and MUR distributions on the forgetting set $(x, y^-, y^+)$1 for each unlearning method and model.

Figure 2: DUR, RUR, and MUR distributions on up-to-date test set $(x, y^-, y^+)$2, measuring generalization and practical impact.

Deprecated API Suppression

All unlearning approaches significantly reduce DUR compared to pre-unlearning baselines. Notably, token-level suppression (PROD) achieves stronger relative reduction than sequence-level (SimNPO) on most models, with CURE variants depressing DUR to even lower levels (down to 4.2% on StarCoder2).

Replacement-Oriented Behavior

Suppression-only methods (PROD, SimNPO) often increase MUR by encouraging avoidance rather than alignment—models simply avoid both deprecated and updated APIs. The incorporation of the contrastive loss in CURE drives RUR up and MUR down, indicating a behavioral shift towards correct replacements rather than generic avoidance.

Generalization to Up-to-date Contexts

On $(x, y^-, y^+)$3, all methods further lower DUR, but only CURE substantially raises RUR and lowers MUR, validating its replacement-focused alignment even in unseen scenarios.

Utility Preservation

HumanEval $(x, y^-, y^+)$4 metrics report negligible degradation post-unlearning; CURE variants consistently maintain or improve baseline performance relative to suppression-only methods for both Pass@1 and Pass@5, indicating that localized knowledge editing does not compromise the overall functional competence of the LLM. The best results across nearly all metrics are achieved with CURE.

Discussion and Theoretical/Practical Implications

CURE advances machine unlearning in code LLMs conceptually and practically:

• Theoretical implication: The explicit contrastive mechanism renders the knowledge update more than "deletion", effecting transitive knowledge alignment in a controlled manner at the model level. It operationalizes the transition from mere catastrophic memory suppression to directed semantic editing.
• Practical implication: Safeguarding code generation reliability in industrial settings becomes tractable without expensive global retraining. Adoption of CURE-like frameworks would ensure forward compatibility with evolving APIs, directly mitigating maintenance burden and security liabilities induced by obsolete API invocations.
• Limitations and Future Directions: The method presupposes the availability of high-quality deprecated-to-updated API mappings and positive context rewriters (e.g., LLMs such as Gemini-2.5-Pro). Future work might investigate integrating retrieval-augmented code generation (RAG) pipelines for discovering replacements dynamically, or extending the methodology to address non-API semantic drift (e.g., algorithmic deprecation, security-relevant patches).

Conclusion

CURE delineates a principled, contrastive approach to knowledge alignment in Code LLMs under API evolution, outperforming suppression-centric edits in both specific behavioral targets (API migration) and overall utility retention. By coupling unlearning and replacement-centric contrastive learning, the approach brings parametric editing closer to the demands of real-world software maintenance, with implications for lifelong knowledge management in generative code models. The released replication package facilitates further advances and benchmarking in this domain.