---
title: Agentic & Inference-Time Unlearning
url: https://www.emergentmind.com/topics/agentic-and-inference-time-unlearning
type: topic
---

# Agentic & Inference-Time Unlearning

Agentic and inference-time unlearning encompass a suite of emerging methods for removing, suppressing, or neutralizing specific knowledge or memorized content from machine learning models—especially large language models (LLMs)—without updating model weights. These strategies leverage either auxiliary verifier models, multi-agent decision-making, or serving-aware system coordination to enforce “right to be forgotten” (RTBF), privacy, or compliance constraints at runtime. They contrast with classical approaches that require full or partial retraining on a reduced dataset, offering a tractable and scalable solution in scenarios where model access, compute, or safety requirements prohibit or discourage post hoc weight modification.

## 1. Overview and Motivation

Machine unlearning addresses the challenge of excising sensitive or undesirable information from machine learning models after training. Traditional techniques revolve around retraining models or applying fine-tuned, parameter-level interventions, which can be computationally prohibitive and detrimental to general model performance. Furthermore, increasingly strict privacy regimes (such as GDPR and CCPA) and security concerns have necessitated methods that provide robust guarantees while accommodating black-box deployment scenarios.

Inference-time unlearning shifts the unlearning burden from training to deployment by interposing specialized runtime policies—either verifier-driven loops or “agentic” control structures—on model outputs. Agentic unlearning systems recast unlearning as a multi-agent, sequential decision problem, where specialized components orchestrate, audit, and sanitize model responses to ensure that sensitive content is not revealed, even under adversarial prompting or distributional shifts. This paradigm provides both operational flexibility and empirical guarantees regarding prompt-level safety, utility preservation, and computational tractability [2602.03787][2502.00406][2504.20965][2311.16136].

## 2. Mathematical Formulation and Guarantees

### Probabilistic Guarantees and Unlearning Error

A central mechanism in inference-time unlearning is the use of explicit verification or scoring functions:
- Let $V(y; x)$ be an auxiliary verifier that scores model response $y$ to prompt $x$; higher values indicate greater “forgetfulness.”
- Define a threshold $\lambda$ such that $V(y; x) \geq \lambda$ signals a safe (non-leaking) answer.

The unlearning error is then
\[
\text{error} = P[ V(y ; x) < \lambda ]
\]
and the target is to drive this probability below a user-specified level $\alpha$, i.e., coverage guarantee $P[ V(y ; x) \geq \lambda ] \geq 1 - \alpha$.

### Conformal Calibration for Distribution-Free Bounds

Inference-time unlearning frameworks (notably, conformal unlearning) calibrate an iteration bound $T_\alpha$ using a calibration set $\mathcal{D}_\text{cal}$ of $m$ i.i.d. prompts. For each $X_i$, the system records the attempt count $T_i$ to achieve $V(y ; X_i) \geq \lambda$. The critical conformal iteration threshold is set as:
\[
T_\alpha = \text{Quantile}\left( \{ T_i \}; \frac{ \lceil (m+1)(1 - \alpha) \rceil }{ m + 1 } \right)
\]
By split-conformal theory, running the runtime loop for at most $T_\alpha$ steps guarantees marginal coverage of at least $1 - \alpha$ [2602.03787].

### Agentic Unlearning Objective

Formalizations adopted by agentic systems, such as AegisLLM, encode unlearning and retention properties via KL-divergence metrics:
\[
\Delta_f(x) = D_\mathrm{KL}[ P_\theta(\cdot|x) \| P_\text{rand}(\cdot) ]
\]
where $P_\theta(\cdot|x)$ is the actual model’s output and $P_\text{rand}(\cdot)$ a benign (random or deflecting) baseline. The system aims to keep $\Delta_f(x) \leq \epsilon_\text{unlearn}$ for restricted prompts and preserve nominal accuracy for safe prompts [2504.20965]. No model parameters are updated; all guarantees are enforced at inference.

## 3. System Architectures and Algorithms

### Single-Verifier Loop with Conformal Prediction

The verifier-based paradigm involves iteratively generating candidate responses, scoring them, and accepting only those above $\lambda$, with stopping determined by conformal calibration:
- At each step, the LM generates $y_t \sim LM(\cdot | x, \mathcal{H})$ given prompt $x$ and history $\mathcal{H}$.
- $y_t$ is accepted if $V(y_t ; x) \geq \lambda$, else the loop continues up to $T_\alpha$ rounds.
- If no safe answer is produced, output the highest-scoring candidate found [2602.03787].

Time complexity is $O(T_\alpha)$ LM and verifier calls per prompt, with $T_\alpha$ determined by small calibration sets; the method remains strictly retrain-free.

### Agentic Multi-Agent Orchestration

Agentic frameworks (e.g., ALU, AegisLLM) enact a sequential, multi-role pipeline. The canonical architecture features:

| Agent Role     | Input/Function                            | Output/Contribution       |
|----------------|-------------------------------------------|--------------------------|
| Orchestrator   | User query $x$                            | Flags safety/routing     |
| Responder      | $x$ (safe)                                | Candidate response $r$   |
| Evaluator      | $(x, r)$                                  | Assesses for leaks       |
| Deflector      | $x$, response type (if unsafe)            | Refusal/sanitization     |

- Queries pass through orchestrator gating, with unsafe or ambiguous cases diverted to a deflector for non-informative or random responses.
- Candidate answers are further audited and scored; for content flagged as risky, fallback responses are triggered.
- Prompts for each agent are optimized via Bayesian algorithms such as DSPy, maximizing reward functions that target both safety and utility [2504.20965].
- ALU expands this by introducing an AuditErase agent for granular, chain-of-thought erasure and a Composer agent for utility-preserving synthesis [2502.00406].

This workflow does not involve weight modification but achieves constant per-request runtime by capping agentic inference rounds.

### Inference-Time Unlearning in MLaaS Systems

Inference-serving-aware designs (e.g., ERASER) dynamically interleave unlearning execution with inference serving by certifying, on a per-prompt basis, whether unlearning would actually affect a prediction:

- ERASER computes certified vote-margin conditions over SISA-trained model shards to decide if an incoming query can be answered immediately.
- If any certification fails, retraining/unlearning is triggered according to a configurable schedule (immediate, threshold-triggered, or uncertification-triggered) [2311.16136].

This approach separates the timing of unlearning from inference, providing strong privacy guarantees with minimal latency impact.

## 4. Performance, Benchmarks, and Empirical Evaluation

Inference-time and agentic unlearning frameworks have been extensively evaluated on realistic and adversarial benchmarks:

| Benchmark         | Unlearning Focus             | Characteristic Result      |
|-------------------|-----------------------------|---------------------------|
| RWKU              | QA on 200 real-world entities | 93% reduction in error (conformal) |
| WPU               | Wikipedia biographies (100 forget/100 retain) | Utility maintained (Δ ≤2%)|
| WMDP              | Sensitive STEM, multi-choice | WMDP acc. at random (ALU/AegisLLM)|
| MMLU              | Retain: college-level MCQ    | Retain acc. loss ≤1–2%     |
| TOFU              | Fictional authors (synthetic/unlearning) | ROUGE-L dropped to 0.057 (ALU) |

- Conformal unlearning reduced unlearning error by up to 93% versus vanilla generation and outperformed Best-of-N, greedy sampling, and parameter-optimization baselines [2602.03787].
- ALU achieved near-zero Forget-ROUGE and retained 90–98% of utility on retain sets; scaled robustly up to 1,000 targets with $O(1)$ latency increase [2502.00406].
- AegisLLM reached WMDP accuracy matching random guessing with only 20 training examples and $<300$ LM calls; retain performance on MT-Bench fell by only 0.4 points [2504.20965].
- ERASER delivered up to $1,290\times$ inference latency speedups (DIMP variant) and reduced retraining events by 30% (threshold-triggered), compared to inference-oblivious baselines [2311.16136].

## 5. Trade-offs, Robustness, and Theoretical Insights

Several design trade-offs and robustness considerations govern the development and evaluation of these frameworks:

- **Efficacy-Utility Trade-off:** Overly aggressive deletion can induce “catastrophic forgetting” of unrelated content. Multi-agent and critic-driven filtering strategies directly optimize the balance between suppression and informativeness [2502.00406].
- **Black-box Compatibility:** All leading agentic and verifier-loop solutions require only inference API access. There are no model weight updates, satisfying practical privacy, legal, and deployment constraints.
- **Runtime Scalability:** Multi-agent and conformal approaches achieve constant-time or predictably bounded runtime per query, regardless of forget set size [2602.03787][2502.00406].
- **Resistance to Jailbreaking:** Robustness to adversarial prompting, target-masking, multilingual paraphrase, and in-context attack chains is empirically substantiated, with agentic frameworks like ALU and AegisLLM outperforming static keyword guardrails and optimization-based retraining [2502.00406][2504.20965].

Theoretical claims center on distribution-free coverage guarantees (split-conformal bounds), constant-time inference (fixed agent call graphs), and agentic rationales for scheduling and threshold optimization [2602.03787][2311.16136].

## 6. Extensions to Agentic and Adaptive AI

Agentic inference-time unlearning is both a methodology and a template for broader “smart forgetting” systems:

- **Dynamic Resource-Aware Scheduling:** ERASER and similar serving-aware systems enable dynamic balance between retraining cost and privacy risk, adaptive to server load and resource constraints [2311.16136].
- **Prompt-Space Adaptivity:** Multi-agent systems can tune or evolve their guard- and response routines in response to evolving threats or task distributions through automated prompt optimization (e.g., via DSPy/Bayesian search in AegisLLM) [2504.20965].
- **Type-Universal Scalability:** Agentic unlearning approaches extend naturally to multimodal and retrieval-augmented architectures, and can be positioned as intermediaries for federated or cloud-based AI APIs [2502.00406].

A plausible implication is that as autonomous and interactive AI expands in prevalence, the principles underlying agentic, inference-time unlearning will become foundational in designing regulatory-compliant, privacy-resilient AI services.

## 7. Limitations and Prospects

Although these frameworks achieve significant practical and empirical results, certain limitations persist:
- Over-suppression in small models and under long forget lists (as seen with 3B-parameter LLMs in ALU) can impair generalization [2502.00406].
- Security assumptions regarding agent prompt integrity and privacy remain areas for continued examination.
- Formal verification of robustness to global adversarial in-context attacks and extension to continuous learning scenarios are open challenges.

Ongoing research directions include development of lightweight critic models, adaptive candidate sampling schemes, multimodal system integration, and formal guarantees under online or federated forgetting demands.

---

**Key References:**  
- "Inference-time Unlearning Using Conformal Prediction" [2602.03787]  
- "ERASER: Machine Unlearning in MLaaS via an Inference Serving-Aware Approach" [2311.16136]  
- "AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security" [2504.20965]  
- "ALU: Agentic LLM Unlearning" [2502.00406]

Source: https://www.emergentmind.com/topics/agentic-and-inference-time-unlearning