SISA Framework for Machine Unlearning

• SISA Framework is a method that partitions training into shards and slices to localize data influence, enabling efficient machine unlearning without full retraining.
• It reduces computational costs by retraining only the affected model component from a specific checkpoint, achieving measurable speed-ups in experimental evaluations.
• The framework supports regulatory compliance and scalability, with extensions addressing fairness, domain-specific applications, and enhanced auditing mechanisms.

The SISA (Sharded, Isolated, Sliced, Aggregated) framework is a machine unlearning methodology designed to efficiently expunge the influence of specified data points from machine learning models without having to retrain models from scratch. It was introduced to reconcile the tension between modern data protection regulations—such as the right to be forgotten—and the practical computational constraints of large-scale ML systems. By compartmentalizing the impact of individual data points during training and providing mechanisms for efficient, targeted retraining, SISA has emerged as a foundational technique with concrete guarantees suitable for regulatory compliance and scalable deployment.

1. Foundations and Structure of the SISA Framework

SISA’s core innovation lies in partitioning the training process so that the effect of each data point is highly localized. This is achieved through four main structural components:

1. Sharding: The training dataset $\mathcal{D}$ is divided into $S$ disjoint shards ($\mathcal{D} = \bigcup_{k=1}^{S} \mathcal{D}_k$ with $\mathcal{D}_k \cap \mathcal{D}_l = \emptyset$ for $k \neq l$). Each data point appears in only one shard.
2. Isolation: Constituent models are trained separately on each shard; no gradients or learned parameters are shared. This guarantees that the influence of each data point is restricted to a single constituent.
3. Slicing: Within each shard, data is further partitioned into $R$ sequential slices ($$\mathcal{D}_k = \bigcup_{i=1}^{R} \mathcal{D}_{k,i}$$). Training proceeds slice by slice with model checkpoints saved after each slice. This enables retraining from an intermediate state rather than the initial state.
4. Aggregation: At inference, predictions from all constituent models are combined (e.g., via majority vote or averaging of prediction vectors) to yield the final output.

This structure ensures that an unlearning request—corresponding to any given data point—requires updating only a single constituent model and typically only from a specific slice checkpoint, rather than retraining the entire model on all data.

2. Unlearning Process and Computational Guarantees

When a user requests removal of their data point $d_u$, SISA only retrains the affected constituent starting from the checkpoint immediately before $d_u$ was introduced. If $d_u$ lies in shard $k$, slice $r$, retraining is triggered from the $(r-1)$-th checkpoint in model $k$ neglecting $d_u$. This yields the following efficiency properties:

• The expected retraining cost per deletion is approximately $\frac{N}{S}$ data points, assuming $N$ is the total training set size and $S$ the number of shards—much less than the full $N$ of retraining.
• Slicing further reduces the retraining workload by allowing rollback to a mid-training state rather than the beginning of the constituent model's training.

Formally, for batch unlearning (processing $K$ deletion requests), the expected number of retrained points is

$\mathbb{E}[C] = N\cdot[1-(1-1/S)^K] - K$

which reflects additional savings by not retraining unaffected shards.

3. Experimental Evaluation and Empirical Performance

SISA was validated across simple and complex tasks to assess both computational savings and potential accuracy degradation:

• Purchase and SVHN datasets (simple tasks): With $S=20$ shards and $R=50$ slices, SISA achieved a $4.63\times$ speed-up (Purchase) and $2.45\times$ speed-up (SVHN) relative to retraining from scratch, while maintaining comparable accuracy.
• ImageNet classification (complex task): Speed-up was more modest ($1.36\times$), with some accuracy loss (~19.45 percentage point drop in top-5 accuracy with naive partitioning). To mitigate this, SISA used transfer learning: pretraining a global model and applying SISA during fine-tuning, closing the accuracy gap to within 4 percentage points for CIFAR-100 (top-1 accuracy).

Further, the computational benefits scale inversely with the number of shards; however, aggressive partitioning can degrade model accuracy due to smaller effective training sets per constituent.

4. Advanced Partitioning Strategies and Cost Optimization

In settings where the distribution of unlearning requests is non-uniform (e.g., data from certain jurisdictions or users is likely to be deleted), SISA supports distribution-aware sharding:

• Groups likely to be subject to deletion are packed into dedicated shards. On unlearning, only a minority of shards are impacted, further reducing retraining volumes.
• The algorithmic approach is to sort examples by their estimated unlearning probability $p(u)$ and pack into shards up to a desired probability threshold.
• In experiments, this approach reduced unlearning cost at the price of minor accuracy losses (e.g., a 1-point accuracy drop in some aggregate predictions).

5. Aggregation, Fairness, and Extensions

At inference, SISA aggregates predictions from constituent models. While majority voting or arithmetic averaging are common, ensemble post-processing can be employed to improve fairness—particularly for tasks like LLM classification. Works such as FairSISA extend post-processing techniques (e.g., based on equalized odds constraints) to SISA ensembles, by solving linear programs to set post-hoc label probabilities and thereby improve group fairness metrics (Kadhe et al., 2023).

Other extensions include:

• SISA-A and SISA-FC: Adapter-based and fully connected head-based SISA implementations for NLP tasks (Kumar et al., 2022). These variants substantially reduce retraining and checkpointing costs for large transformer architectures.
• Applications in specialized domains: Modifications of SISA for graph neural networks (Chen et al., 2021) and adaptation to split learning setups (Yu et al., 2023) demonstrate its flexibility.
• SISA++: A refinement for speech tasks using weight averaging of constituent models, further stabilizing accuracy post-unlearning (Phukan et al., 2 Jun 2025).

6. Limitations, Security, and Practical Considerations

Despite its advantages, SISA is not without limitations:

• Accuracy–Efficiency Trade-off: Excessive partitioning can harm accuracy by reducing the amount of data seen by any constituent model, especially on complex tasks. Mitigation requires transfer learning or careful tuning of shard counts.
• Security Risks: The SISA structure can be exploited in backdoor attacks; adversaries can plant data and then issue unlearning requests to activate dormant backdoors (Zhang et al., 2023). Increasing the number of shards makes such attacks more challenging, but can reduce detection effectiveness.
• Auditability: SISA’s design provides a “certificate” of unlearning, assuming proper retraining traces exist. Tools such as TruVRF (Zhou et al., 12 Aug 2024) can non-invasively verify whether unlearning requests were faithfully executed by inspecting post-unlearning model sensitivities at class, sample, and volume levels.
• Fairness: SISA generally outperforms naive retraining and other unlearning mechanisms under non-uniform deletions in terms of group fairness but sometimes requires post-processing corrections as mentioned above.

7. Practical Deployment and Future Directions

SISA provides a scalable and operationally tractable solution for organizations facing frequent data deletion requests. Its design tightly aligns with data protection obligations under legal frameworks such as GDPR and CCPA. In deployment:

• Service providers can balance the number of shards and slices to control retraining costs and accuracy.
• By employing distribution-aware strategies and transfer learning, SISA can be adapted for both simple and complex modeling pipelines.
• Post-processing and auditing frameworks can be combined with SISA to verify fairness and effective forgetting.
• SISA’s methodology is being extended to integrate continual learning, knowledge distillation, and parameter-efficient fine-tuning, suggesting possible future directions for unified learning and unlearning systems.

SISA’s compartmentalization of influence—by partitioning datasets and isolating training—establishes a practical paradigm for efficient, provable, and governable machine unlearning, cementing its role in the implementation of trustworthy, privacy-centric machine learning systems.