CORTIS: Cumulative ORThogonal Identity Suppression

• CORTIS is a continual unlearning framework for zero-shot TTS that sequentially erases speaker identities without accessing previous forget data.
• It employs contrastive Fisher-information-based parameter masking to localize forgetting to speaker-relevant weights and maintain overall quality.
• The method uses orthogonal projection to prevent catastrophic re-learning, ensuring that past unlearning effects remain intact during new updates.

Searching arXiv for the cited CORTIS papers to ground the article and disambiguate the term. Cumulative ORThogonal Identity Suppression (CORTIS) is a continual machine-unlearning framework for zero-shot text-to-speech (ZS-TTS) models that is designed to erase speaker identities over a sequence of “right to be forgotten” requests while requiring no access to previously unlearned speaker data and avoiding the reappearance of previously removed voices through later updates. In the formulation introduced for VoiceBox, CORTIS combines Fisher-information-based parameter masking, which localizes unlearning to speaker-relevant weights, with orthogonal projection against subspaces spanned by prior unlearning updates, so that new requests can be processed without replaying old forget-speaker data (Kim et al., 25 May 2026).

1. Formal setting in continual speaker identity unlearning

In ZS-TTS, a pre-trained model $\theta_0$ generates speech that matches an arbitrary target speaker from a short reference prompt $x^s$ and a text transcription $y$, so that

$\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$

Speaker identity is not isolated in a single module; it is entangled in parameters that process text, audio context, and style. Speaker identity unlearning modifies the model so that, for a requested speaker $f$, the system can no longer reproduce that voice,

$\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$

while preserving ordinary behavior for other speakers $r$,

$\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$

The continual setting formalizes a deployed ZS-TTS service that receives requests over time rather than as a single batch. Let $\mathcal{S}$ denote the speaker set and $\mathcal{D}^s=\{(x^s,y)\}$ the data for speaker $x^s$0. At step $x^s$1, a new forget request $x^s$2 arrives, the cumulative forget set becomes

$x^s$3

and the remain set is

$x^s$4

The model is updated from $x^s$5 to $x^s$6 without access to future requests.

Two deployment constraints are central. First, requests arrive sequentially. Second, forget-speaker data must not be retained after the corresponding request has been processed. Accordingly, at step $x^s$7 the system may use the current model, the current forget-speaker data $x^s$8, and remain data $x^s$9 or a subset thereof, but it has no access to past forget data $y$0 and cannot retrain from scratch on the cumulative forget set. The stepwise objective is therefore to maintain

$y$1

2. Catastrophic re-learning as the central failure mode

The paper distinguishes continual speaker unlearning from standard continual learning by identifying a different failure mode. Rather than catastrophic forgetting of prior tasks, the salient risk is catastrophic re-learning: a speaker that was previously removed becomes clonable again after the model is updated to forget a new speaker (Kim et al., 25 May 2026).

This behavior arises because prior speaker-unlearning methods, such as Teacher-Guided Unlearning (TGU), combine a forget loss for the current speaker with a strong retain loss intended to preserve general ZS-TTS capability. When a new speaker $y$2 is processed, the forget loss applies only to $y$3; earlier forget speakers are no longer directly constrained. The retain loss, however, still encourages generic voice-cloning quality. Because ZS-TTS models generalize across speakers, that generic pressure can inadvertently restore the ability to mimic earlier forgotten voices.

The reported sequential behavior of TGU makes this failure concrete. After unlearning $y$4, speaker similarity for that speaker is low, indicating successful forgetting. After then unlearning $y$5, the newly targeted speaker is again suppressed, but the similarity for $y$6 rises sharply toward the original model’s level. The same pattern repeats at later requests. This creates what the paper terms a data retention paradox: durable forgetting would be straightforward if one could replay all prior forget speakers at every step, but replay would require retaining the very data that “right to be forgotten” processing is supposed to discard.

CORTIS is constructed specifically for this setting. Its objective is not merely to reduce parameter drift, but to preserve the unlearning effect for all speakers in $y$7 while processing $y$8 without storing old forget data or checkpoints that are effectively equivalent to storing their data.

3. Contrastive Fisher-information-based parameter masking

The first mechanism in CORTIS is contrastive Fisher-information-based parameter masking. For a loss $y$9, the empirical diagonal Fisher for parameter $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$0 on dataset $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$1 is defined as

$\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$2

At unlearning step $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$3, Fisher diagonals are computed for the current forget set $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$4 and for the remain set or a fixed subset thereof. Fisher statistics from earlier forget steps are stored as per-parameter scalar importance values rather than as raw data.

CORTIS then forms a contrastive saliency map,

$\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$5

with the max taken elementwise. The numerator measures importance for forgetting the current speaker, while the denominator penalizes parameters important either for remain-set quality or for earlier forget speakers. Parameters with high saliency are therefore those that are highly useful for the new forget objective but relatively unimportant to retention and prior unlearning.

A binary mask $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$6 is constructed by selecting the global top-$\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$7 of saliency values. Only those parameters are trainable during request $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$8; all other weights are frozen. The main setting uses $\theta_0(x^s, y) \approx \hat{x}^{spk=s}_y.$9, so $f$0 of parameters remain frozen. The masked update has the form

$f$1

The role of this masking is twofold. First, it localizes the current unlearning step to a restricted parameter subset. Second, by explicitly comparing current forget importance against both retain importance and stored past forget importance, it aims to avoid modifying the core parameters that support prior forgetting or remain-set quality. The paper reports that Jaccard overlaps between masks for different speakers are small, with values below $f$2, which is presented as evidence that different speakers tend to occupy near-disjoint parameter subsets.

4. Orthogonal projection and cumulative forget subspaces

Masking alone does not constrain the direction of updates inside the trainable subset. CORTIS therefore adds a second mechanism: orthogonal projection in update space, inspired by Orthogonal Gradient Descent and Gradient Projection Memory (Kim et al., 25 May 2026).

For each unlearning sequence, masked gradient snapshots $f$3 are collected periodically during optimization and stacked as columns in a matrix

$f$4

where $f$5 is the number of masked parameters. Before singular value decomposition, each gradient is made orthogonal to the previously accumulated basis:

$f$6

A truncated SVD is then computed,

$f$7

and the top-$f$8 left singular vectors in $f$9 define the principal directions used to forget speaker $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$0 within the masked subspace. The main experiments use $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$1.

To avoid an ever-growing concatenation of per-speaker bases, CORTIS maintains a fixed-rank merged basis. It forms the energy-weighted stack

$\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$2

computes a rank-$\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$3 truncated SVD, and uses the resulting $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$4 as the cumulative forget subspace basis. This compresses prior forgetting directions into a fixed-dimensional representation.

During later requests, a new masked update $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$5 is projected onto the orthogonal complement of that cumulative basis:

$\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$6

The projected update is then applied to the masked parameters. In the paper’s geometric interpretation, this removes any component that moves along directions previously used to forget earlier speakers. As a consequence, future optimization is prevented from “walking back” along prior forgetting directions.

The per-request workflow is therefore: compute Fisher for the current forget speaker, build the saliency mask, optimize the base forget-plus-retain objective with masked and projected gradients, collect gradient snapshots, derive a per-speaker basis by truncated SVD, merge it into the cumulative basis, and then discard the raw forget-speaker data. Persistent state consists of the current model, stored Fisher diagonals, and the low-rank basis information.

5. Empirical behavior on VoiceBox

The experiments use VoiceBox, a 24-layer Transformer trained for 500k steps on LibriHeavy, with five LibriHeavy speakers as forget targets, each with approximately 20 minutes of audio (Kim et al., 25 May 2026). The forget sequence is reported as speaker IDs $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$7. Remain-set evaluation is conducted on LibriSpeech test-clean.

Two metric families are used. Word Error Rate is measured on remain and forget sets with HuBERT-Large fine-tuned on LibriSpeech as the ASR backend. Speaker similarity is computed as cosine similarity between WavLM-TDCNN embeddings of the prompt and generated audio. The paper calibrates similarity thresholds using real same-speaker and different-speaker distributions on LibriSpeech: the different-speaker upper whisker is approximately $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$8, and the same-speaker lower whisker is approximately $\theta(x^f, y) \not\approx \hat{x}^{spk=f}_y,$9. Under this calibration, $r$0-$r$1 indicates a failure of retain quality, whereas $r$2-$r$3 indicates a failure of forgetting.

After three requests, the original model has $r$4-$r$5 and high forget-speaker similarity values, including $r$6-$r$7, $r$8-$r$9, and $\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$0-$\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$1. CORTIS yields $\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$2-$\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$3–$\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$4, $\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$5-$\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$6, $\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$7-$\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$8, and low forget-speaker similarities: $\theta(x^r, y) \approx \hat{x}^{spk=r}_y.$9-$\mathcal{S}$0, $\mathcal{S}$1-$\mathcal{S}$2, and $\mathcal{S}$3-$\mathcal{S}$4. These values place retain performance above the paper’s retain threshold while keeping all forget speakers in the forget regime.

The comparison with baselines is central to the method’s empirical characterization. Sequential TGU preserves retain performance but exhibits catastrophic re-learning: after Request 1, $\mathcal{S}$5-$\mathcal{S}$6; after Request 2, $\mathcal{S}$7-$\mathcal{S}$8; and after Request 3, $\mathcal{S}$9-$\mathcal{D}^s=\{(x^s,y)\}$0 while $\mathcal{D}^s=\{(x^s,y)\}$1-$\mathcal{D}^s=\{(x^s,y)\}$2. Sample-Guided Unlearning avoids re-learning but collapses retain quality, with $\mathcal{D}^s=\{(x^s,y)\}$3-$\mathcal{D}^s=\{(x^s,y)\}$4 falling to $\mathcal{D}^s=\{(x^s,y)\}$5 by Request 3. Update Norm and Selective Fine-tuning preserve retain similarity at roughly $\mathcal{D}^s=\{(x^s,y)\}$6–$\mathcal{D}^s=\{(x^s,y)\}$7 after three requests, but they do not prevent re-learning: after Request 3, $\mathcal{D}^s=\{(x^s,y)\}$8-$\mathcal{D}^s=\{(x^s,y)\}$9 remains around $x^s$00–$x^s$01.

The five-request sequence extends the same pattern. The reported $x^s$02-$x^s$03 remains around $x^s$04–$x^s$05 across requests, while the maximum forget-speaker similarity at each step stays below approximately $x^s$06. The oldest forgotten speaker does not drift upward over time. An ablation further shows that masking alone is insufficient: without projection, after three requests the model reaches $x^s$07-$x^s$08 and $x^s$09-$x^s$10, indicating partial but incomplete protection against re-learning.

The computational profile is also reported. Sequential TGU uses 10k steps per request and approximately 29 hours per request. TGU on the cumulative forget set, which is not RTBF-compliant, takes approximately 87.5 hours after three requests as training cost grows with the number of forget speakers. CORTIS uses only approximately 3k steps per request, requires about 3.5 hours per request on $x^s$11 A100 80GB, and introduces a projection overhead of approximately $x^s$12 per step.

6. Conceptual significance, limitations, and future directions

CORTIS is positioned as a continual conceptual unlearning method that combines regularization-based and optimization-based ideas. The Fisher mask is described as a continual-unlearning analogue of hard importance-based protection, while the projection mechanism enforces an explicit geometric constraint on future updates. A central conceptual claim of the paper is that durable unlearning requires protecting both parameters and directions: limiting parameter drift alone is not sufficient, because a model can still move within the subspace spanned by earlier forgetting directions and thereby restore prior functionality (Kim et al., 25 May 2026).

The method is presented without formal guarantees on similarity or retention. Instead, the argument is geometric: if prior forgetting is captured by the stored subspace, and future updates are projected to be orthogonal to that subspace, then those earlier forgetting directions cannot be undone by later optimization steps. This suggests that subspace quality is critical to practical durability.

Several limitations are explicit. Experiments are restricted to VoiceBox, so architectural generality remains unverified. The fixed-rank merged basis may become an increasingly coarse approximation as the number of requests grows, which could eventually lead either to leakage of previously forgotten identities or to excessive constraints on later learning. The method assumes an honest service provider and does not study adversarial fine-tuning or prompt-based recovery attacks. Storing Fisher diagonals and low-rank bases is described as modest but nonzero overhead, particularly for very large models or long request sequences.

The future directions suggested in the paper follow directly from these limits: extending the approach to other generative domains, studying stronger privacy guarantees and adversarial robustness, and exploring alternative subspace tracking or non-linear constraints for more complex forgetting dynamics.

7. Terminological disambiguation

The acronym “CORTIS” is not unique on arXiv. In the continual speaker-unlearning work, it denotes Cumulative ORThogonal Identity Suppression and refers to the combination of contrastive Fisher masking with orthogonal projection for sequential speaker identity removal in ZS-TTS (Kim et al., 25 May 2026).

A separate paper, “CORTIS: Text-Only Adaptation of Spoken LLMs for Task-Oriented Voice Agents,” uses the same acronym for COnveRsational agents via Text-only supervised Instruction tuning of SLMs and explicitly states that it is not “Cumulative ORThogonal Identity Suppression” (Choi et al., 19 Jun 2026). That work concerns text-only adaptation of spoken LLMs for structured-output voice agents, with frozen speech modules and standard next-token prediction over serialized outputs. It defines no cumulative, orthogonal, or identity-suppression loss. The two methods therefore share only an acronym; they address different modalities, objectives, and technical mechanisms.