Rotation Control Unlearning (RCU)

• Rotation Control Unlearning (RCU) is a method that employs cognitive rotations via skew-symmetric LoRA updates to achieve controlled forgetting in LLMs.
• It uses a rotational salience weight to scale rotation angles, enabling continuous and precise unlearning under sequential requests.
• RCU avoids the need for retained datasets and mitigates cumulative utility loss, effectively preserving performance on benign data.

Searching arXiv for the named method and closely related projection/LoRA unlearning work. Rotation Control Unlearning (RCU) is a machine unlearning method for LLMs that studies unlearning in a realistic, continuous setting, where a model receives multiple, sequential requests to forget different data or concepts. It represents each unlearning operation as a rotation in a “cognitive rotation space” defined by skew-symmetric LoRA updates, uses the rotation angle as a quantitative measure of degree of forgetting, and introduces a rotational salience weight $\beta$ to control unlearning continuously for each input. The method is designed to avoid dependence on retained datasets and to mitigate cumulative catastrophic utility loss under continuous unlearning requests (Zhang et al., 30 Sep 2025).

1. Problem formulation and motivation

RCU is formulated for a causal LLM $M_\Theta$ with parameters $\Theta$ that receives a sequence of unlearning requests indexed by $t = 1,\dots,T$. The $t$-th request is associated with an unlearning dataset

$$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$

with input distribution $\mathcal{P}_{\mathcal{X}^{U,t}}$ and corresponding label distribution $\mathcal{P}_{\mathcal{Y}^{t}}$ (Zhang et al., 30 Sep 2025).

The method is motivated by a limitation of classical LLM unlearning procedures. Existing methods often assume access to a retained dataset drawn from a distribution $\mathcal{P}_{\mathcal{X}^{R,t}}$ disjoint from $\mathcal{P}_{\mathcal{X}^{U,t}}$, and use that retained dataset to keep utility intact while forgetting. The RCU formulation emphasizes that, for LLMs, training data is huge, using a sufficiently representative retained subset is impractical, and in continuous unlearning, reusing large retained sets for every request is computationally prohibitive (Zhang et al., 30 Sep 2025).

The continuous setting is central. New privacy requests, new illegal content categories, or new safety policies may appear over time, so the operational object is a sequence $M_\Theta$0, not a single one-shot batch. In this setting, many parameter-based unlearning methods are described as causing the model to drift further from its original behaviors with each new unlearning step, so utility on benign data distributions $M_\Theta$1 can degrade catastrophically and in an accumulating fashion (Zhang et al., 30 Sep 2025).

The immediate goals are expressed as forgetting on unlearning distributions while preserving utility on benign distributions. The forget objective is written as

$M_\Theta$2

while the utility-preservation objective is

$M_\Theta$3

Within the paper’s framing, unlearning is motivated by removing private data memorized by LLMs, erasing illegal, toxic, or harmful content, and ensuring models can be retrofitted to updated policies without full retraining (Zhang et al., 30 Sep 2025).

2. Cognitive rotation space and controlled forgetting

RCU uses LoRA for a weight matrix $M_\Theta$4, introducing low-rank matrices $M_\Theta$5 and $M_\Theta$6, with

$M_\Theta$7

Where traditional LoRA uses

$M_\Theta$8

RCU reinterprets the update as

$M_\Theta$9

and constrains $\Theta$0 to be skew-symmetric so that $\Theta$1 approximates a rotation (Zhang et al., 30 Sep 2025).

This construction is embedded in Lie group and Lie algebra language. The cognitive rotation space $\Theta$2 is treated as an $\Theta$3-dimensional rotation matrix satisfying $\Theta$4 and $\Theta$5, hence $\Theta$6. The formulation uses the fact that every $\Theta$7 corresponds to at least one matrix $\Theta$8 in the Lie algebra $\Theta$9 of skew-symmetric matrices through the exponential map

$t = 1,\dots,T$0

Using the Taylor expansion,

$t = 1,\dots,T$1

RCU identifies the LoRA update matrix $t = 1,\dots,T$2 with such a $t = 1,\dots,T$3, so that if $t = 1,\dots,T$4 is skew-symmetric and small in norm, then

$t = 1,\dots,T$5

and the transformed weights are

$t = 1,\dots,T$6

The paper calls this family of transformations the cognitive rotation space (Zhang et al., 30 Sep 2025).

The degree of forgetting is controlled by the rotational salience weight $t = 1,\dots,T$7, which rescales the LoRA update: $t = 1,\dots,T$8 Here $t = 1,\dots,T$9 defines a direction of rotation in cognitive space for the current unlearning request, while $t$0 controls the rotation angle magnitude and thus the degree of unlearning for a specific input $t$1. The paper states:

Theorem 1. For $t$2, when $t$3, the rotation angle $t$4 of $t$5 is directly proportional to $t$6.

Under this result, changing $t$7 to $t$8 changes the corresponding rotation from $t$9 to $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$0, and the rotation angle scales from $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$1 to $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$2. This is the basis for treating $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$3 as a continuous knob: $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$4 corresponds to no unlearning, $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$5 corresponds to full LoRA unlearning applied, and intermediate values correspond to partial forgetting (Zhang et al., 30 Sep 2025).

A common misconception is that RCU applies exact rotations throughout the model. In the formulation, $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$6 is justified by the assumption that LoRA updates remain small in norm, described empirically as around $$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$7–$$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$8, and only attention layers are rotated. The method therefore provides a controlled rotational approximation rather than a claim of exact global orthogonal transformation of the entire network (Zhang et al., 30 Sep 2025).

3. Loss design and training mechanics

The rotational interpretation is enforced by a skew-symmetric loss

$$D^{U,t} = \{x_i \mid x_i \sim \mathcal{P}_{\mathcal{X}^{U,t}}\}_{i=1}^{N^{U,t}},$$9

If $\mathcal{P}_{\mathcal{X}^{U,t}}$0 is perfectly skew-symmetric, then $\mathcal{P}_{\mathcal{X}^{U,t}}$1, $\mathcal{P}_{\mathcal{X}^{U,t}}$2, and $\mathcal{P}_{\mathcal{X}^{U,t}}$3. Minimizing this loss pushes $\mathcal{P}_{\mathcal{X}^{U,t}}$4 into $\mathcal{P}_{\mathcal{X}^{U,t}}$5, the Lie algebra of skew-symmetric matrices. In the paper’s interpretation, this constructs the cognitive rotation space by making $\mathcal{P}_{\mathcal{X}^{U,t}}$6 a legitimate generator of $\mathcal{P}_{\mathcal{X}^{U,t}}$7, ensuring that $\mathcal{P}_{\mathcal{X}^{U,t}}$8 acts as an approximate rotation, and allowing the LoRA update to be interpreted as a rotation operator in parameter space (Zhang et al., 30 Sep 2025).

The overall optimization objective for the LoRA-based unlearning module is

$\mathcal{P}_{\mathcal{X}^{U,t}}$9

where $\mathcal{P}_{\mathcal{Y}^{t}}$0 is the skew-symmetric loss, $\mathcal{P}_{\mathcal{Y}^{t}}$1 is the orthogonal rotation axes loss, and $\mathcal{P}_{\mathcal{Y}^{t}}$2 is a cross-entropy loss defined over unlearning data $\mathcal{P}_{\mathcal{Y}^{t}}$3 with “refuse” or random answers $\mathcal{P}_{\mathcal{Y}^{t}}$4, for example “I don’t know” (Zhang et al., 30 Sep 2025). On ScienceQA, the reported hyperparameters are $\mathcal{P}_{\mathcal{Y}^{t}}$5, $\mathcal{P}_{\mathcal{Y}^{t}}$6, $\mathcal{P}_{\mathcal{Y}^{t}}$7. On TOFU, they are $\mathcal{P}_{\mathcal{Y}^{t}}$8, $\mathcal{P}_{\mathcal{Y}^{t}}$9, $\mathcal{P}_{\mathcal{X}^{R,t}}$0 (Zhang et al., 30 Sep 2025).

The algorithmic procedure proceeds in stages. The base LLM $\mathcal{P}_{\mathcal{X}^{R,t}}$1 is frozen, LoRA parameters are introduced for each attention layer with rank $\mathcal{P}_{\mathcal{X}^{R,t}}$2 such as $\mathcal{P}_{\mathcal{X}^{R,t}}$3, and only these LoRA parameters are trainable. For each unlearning request $\mathcal{P}_{\mathcal{X}^{R,t}}$4, the method first trains $\mathcal{P}_{\mathcal{X}^{R,t}}$5 by minimizing $\mathcal{P}_{\mathcal{X}^{R,t}}$6. The cross-entropy term drives the model to output refuse or random answers on unlearning data; the skew-symmetric loss forces $\mathcal{P}_{\mathcal{X}^{R,t}}$7 to be skew-symmetric; and the orthogonal rotation axes loss uses the previous LoRA parameters $\mathcal{P}_{\mathcal{X}^{R,t}}$8 to enforce orthogonality between the new rotation axes and past ones when $\mathcal{P}_{\mathcal{X}^{R,t}}$9. This produces a rotation operator $\mathcal{P}_{\mathcal{X}^{U,t}}$0 for the $\mathcal{P}_{\mathcal{X}^{U,t}}$1-th request (Zhang et al., 30 Sep 2025).

4. Continuous unlearning and orthogonal rotation axes

RCU’s distinctive claim is that it is designed explicitly for continuous unlearning. For multiple requests, there are multiple rotations $\mathcal{P}_{\mathcal{X}^{U,t}}$2, and the method seeks to make their axes mutually perpendicular in order to reduce interference (Zhang et al., 30 Sep 2025).

The relative rotation between two successive requests is written as

$\mathcal{P}_{\mathcal{X}^{U,t}}$3

Using skew-symmetry $\mathcal{P}_{\mathcal{X}^{U,t}}$4, this becomes

$\mathcal{P}_{\mathcal{X}^{U,t}}$5

where the second-order term is dropped under the small-update approximation (Zhang et al., 30 Sep 2025).

The geometric basis for the regularizer is stated as:

Theorem 2. When $\mathcal{P}_{\mathcal{X}^{U,t}}$6 and $\mathcal{P}_{\mathcal{X}^{U,t}}$7, and $\mathcal{P}_{\mathcal{X}^{U,t}}$8 (their rotation “faces” or generator matrices are orthogonal), then the rotation axes of $\mathcal{P}_{\mathcal{X}^{U,t}}$9 and $M_\Theta$00 are perpendicular to each other.

The corresponding orthogonal rotation axes loss is

$M_\Theta$01

where $M_\Theta$02 are the LoRA parameters after training on request $M_\Theta$03. Minimizing this term encourages the generators, and hence the rotation axes, to be orthogonal. In the paper’s interpretation, this makes the rotations for different unlearning requests more independent and mitigates cumulative catastrophic utility loss (Zhang et al., 30 Sep 2025).

At inference time, controlled unlearning is implemented per input. An OOD detector computes a score $M_\Theta$04, a distributional shift compensator maps $M_\Theta$05 to $M_\Theta$06, and the model uses the scaled rotation

$M_\Theta$07

which corresponds to rotation angle $M_\Theta$08. For ScienceQA, the mapping uses $M_\Theta$09, $M_\Theta$10, and

$M_\Theta$11

For TOFU, it uses $M_\Theta$12, $M_\Theta$13, and

$M_\Theta$14

These mappings are described as empirical choices that concentrate $M_\Theta$15 in the effective unlearning region, for example $M_\Theta$16 (Zhang et al., 30 Sep 2025).

5. Empirical evaluation, ablations, and reported performance

The evaluation uses two principal settings: ScienceQA for question answering and TOFU for fictitious knowledge generation. The base LLM is LLaMA2-7B, the OOD detector is Pseudo-RoBERTa-Large, the LoRA rank is 8, and only attention layers are fine-tuned. Baselines include GradAsc, GradDif, EUL, PO, NPO, SOGD, SOPO, $M_\Theta$17, and the unmodified base model (Zhang et al., 30 Sep 2025).

For unlearning, the reported metrics are Sample-level Unlearning (S.U.), defined as accuracy on the unlearning training set for a given request, and Distribution-level Unlearning (D.U.), defined as accuracy on the unlearning test set; lower is better for both. For utility preservation, the reported metrics are Retained Distribution (R.D.), CommonsenseQA (C.QA.), OpenBookQA (O.QA.), and on TOFU, Real-world Authors (R.A.) and World Facts (W.F.) (Zhang et al., 30 Sep 2025).

On ScienceQA, RCU is reported to achieve near-base utility: C.QA. and O.QA. accuracies essentially match the base model across all five unlearning requests, and R.D. remains very close to base, often above 90%. Relative to $M_\Theta$18, RCU yields substantially lower S.U. and D.U.: on five unlearning domains, S.U. drops by 9.68%, 4.02%, 4.05%, 7.58%, and 10.82%, and average D.U. is reduced by 17.67%. For the first request, biology, the reported figures are Base: S.U. $M_\Theta$19, D.U. $M_\Theta$20; $M_\Theta$21: S.U. $M_\Theta$22, D.U. $M_\Theta$23; RCU: S.U. $M_\Theta$24, D.U. $M_\Theta$25 (Zhang et al., 30 Sep 2025).

On TOFU, the reported results also favor RCU over the listed baselines. For request 1 (forget01), $M_\Theta$26 gives S.U. $M_\Theta$27, D.U. $M_\Theta$28, while RCU gives S.U. $M_\Theta$29, D.U. $M_\Theta$30. For request 2 (forget05), $M_\Theta$31 gives S.U. $M_\Theta$32, D.U. $M_\Theta$33, while RCU gives S.U. $M_\Theta$34, D.U. $M_\Theta$35. For request 3 (forget10), $M_\Theta$36 gives S.U. $M_\Theta$37, D.U. $M_\Theta$38, while RCU gives S.U. $M_\Theta$39, D.U. $M_\Theta$40. The paper states that R.A. and W.F. stay essentially at base levels for RCU (Zhang et al., 30 Sep 2025).

Parameter efficiency is also part of the reported comparison. The paper states that $M_\Theta$41 uses approximately 19.99M trainable parameters, whereas RCU uses approximately 8.39M trainable parameters through attention LoRA only (Zhang et al., 30 Sep 2025).

The ablation studies isolate the major components. On ScienceQA, removing $M_\Theta$42 causes unlearning essentially to fail, with S.U. and D.U. near 100%. Removing RC-LoRA allows early unlearning but leads to significant utility degradation with more requests, and D.U. escalates, for example above 30%. Removing $M_\Theta$43 preserves some unlearning but utility gradually degrades. Removing $M_\Theta$44 degrades OOD–unlearning alignment and increases variance. The paper’s conclusion is that RC-LoRA, $M_\Theta$45, and $M_\Theta$46 are the three key factors for mitigating cumulative utility loss (Zhang et al., 30 Sep 2025).

6. Relation to projection-based unlearning, limitations, and open questions

RCU belongs to a broader family of geometry-aware unlearning methods, but its mechanism differs from projection-constrained approaches. A close contrast appears in "Null-Space Constrained Low-Rank Adaptation for Response-Specified LLM Unlearning" (Ju et al., 9 Jun 2026), which introduces Null-Space Constrained Response-Specified Unlearning (NSRU). NSRU treats unlearning as a constrained adaptation problem in which low-rank updates act only in the null space of a retain subspace estimated from benign hidden representations. It uses an explicitly structured safe target response, suppresses the original undesired content, and confines LoRA updates through a null-space projection $M_\Theta$47 (Ju et al., 9 Jun 2026).

The comparison clarifies RCU’s distinctive geometry. NSRU uses a projection $M_\Theta$48 on representations feeding LoRA, described as “projection in activation space, realized through special parameterization,” whereas an RCU method uses rotations in representation or weight space to realign harmful versus benign directions, then controls updates in the rotated basis. From a linear algebra standpoint, the supplied description notes that rotations and projections are tightly related: rotations change basis while conserving inner products, and projections such as $M_\Theta$49 implicitly define an orthonormal basis whose first axes span the retain subspace and whose remaining axes span the null space. This suggests that RCU can be viewed as making such a basis explicit via a rotation, while NSRU uses the basis given by $M_\Theta$50 without constructing a full orthogonal matrix (Ju et al., 9 Jun 2026).

This relation also helps delimit the claims of RCU. A common misconception is that geometry alone guarantees forgetting. The NSRU analysis states that geometry alone is not sufficient: without undesired loss, the model may imitate the safe target but still leave forget content extractable; without safe-target loss, there is no alignment with desired behavior; without retain loss, retention collapses (Ju et al., 9 Jun 2026). A plausible implication is that RCU’s geometric control should also be read together with its behavioral objective, namely the cross-entropy loss on unlearning data with refuse or random answers, rather than as a purely geometric deletion mechanism.

The limitations stated for RCU are specific. The method requires access to unlearning datasets $M_\Theta$51 but does not require retained datasets from the original training distribution. It incurs additional cost from training an OOD detector per unlearning phase and running it at inference. The mapping $M_\Theta$52 is dataset-specific and chosen empirically. Only attention layers are rotated; other parts of the model may still encode unwanted knowledge. The paper also states that there is no formal, worst-case guarantee of perfect unlearning, and that combining multiple per-request rotations at inference, if all are active, could be complex because the method implicitly treats them separately via per-request OOD detection (Zhang et al., 30 Sep 2025).

In that sense, RCU is best understood as a geometric framework for controllable, accumulation-resistant unlearning rather than as an unconditional guarantee of erasure. Its principal technical claims are the cognitive rotation space formulation for LoRA updates, the skew-symmetric loss that embeds LoRA updates in $M_\Theta$53, the orthogonal rotation axes regularization for continuous unlearning, and the rotational salience weight $M_\Theta$54 that turns rotation angle into a continuous control variable for forgetting degree (Zhang et al., 30 Sep 2025).