Natively Unlearnable Large Language Models
Abstract: Unlearning aims to remove the influence of specific training data sources, but this has proved challenging because the contributions of different sources are entangled within the model. Isolating source contributions to disjoint parameters makes removal easier, though it obstructs joint learning across sources. We propose NULLs (Natively Unlearnable LLMs), a model class that satisfies the two opposing goals of isolating source-specific contributions and learning jointly across sources, by training a set of shared backbone neurons alongside a pool of sparsely activated sinks. During training, information specific to a source naturally concentrates in its sinks while information shared across sources accumulates in the backbone. A source is then unlearned at deployment by disabling its corresponding sinks, with no gradient updates and no access to the retained data. We show that NULLs scales to Wikipedia's ~6M articles, isolating each as an independent source. Unlearning a single article removes knowledge specific to it while preserving facts shared with semantically related articles, closely matching retraining from scratch. We note that unlearning with NULLs is also robust: in a case study of unlearning the Harry Potter books, NULLs resists both adversarial extraction and relearning that reverses post-hoc unlearning. Finally, NULLs preserves general language capabilities, matching a standard transformer on downstream benchmarks. Together, these results suggest that source-level unlearning need not be an afterthought. It can be built natively into LLM training while retaining the benefits of shared representation learning.
Summary
- The paper introduces a novel transformer architecture (NULLs) that uses deterministic sparse activation masks to enable source-specific data unlearning without retraining.
- The proposed method preserves general knowledge while erasing unique, article-specific facts, matching the performance of retrained-from-scratch models.
- NULLs demonstrates robust resistance to adversarial extraction and rapid relearning attacks, ensuring reliable and permanent data removal.
Natively Unlearnable LLMs (NULLs)
Introduction and Motivation
The paper "Natively Unlearnable LLMs" (2606.13873) formalizes and addresses the challenge of reliable data unlearning in LLMs, a necessity driven by legal, privacy, and ethical obligations. Conventional LLM training intermixes contributions from heterogeneous data, making post-hoc disentanglement or attribution of memorized information infeasible. Existing unlearning approaches face critical trade-offs: post-hoc methods degrade generalization and are vulnerable to adversarial or relearning attacks, while modular or mixture-of-experts approaches preclude joint learning and scale poorly to million-source settings. NULLs is introduced to reconcile these objectives, enabling fine-grained, scalable, and robust unlearning without compromising generalization.
Figure 1: The NULLs architecture enables per-source isolation via sparse neuron activation masks, allowing source-specific information to be unlearned by mask deactivation, while a shared backbone aggregates general knowledge.
NULLs Model Architecture
The core architectural intervention in NULLs is the introduction of a shared backbone neuron set and a large pool of sparsely activated "sink" neurons within each transformer's MLP layer. Each training source (e.g., document, cluster, publisher) is deterministically mapped—via its identifier used as a seed—to a unique binary mask which selects a small subset of sink neurons for activation, while the backbone is always active. This enables encoding source-specific information in an addressable, non-entangled form, while generalizable patterns are absorbed by the backbone, preserving shared representation learning.
Standard transformer training is preserved except for an additional elementwise multiplication by the source mask after the MLP nonlinearity. Sink neuron pool and mask sizes are selected to ensure that, even with millions of sources, mask overlap is low, and source-specific interference is minimal.
Data Unlearning in NULLs
Unlearning any source is implemented by zeroing out (or disabling) that source's mask at inference. This is equivalent to removing its contribution from the model without any retraining, data storage, or gradient updates. Furthermore, permanent erasure is achieved by parameterization: weights associated with sink neurons for a given source can be purged. Crucially, the architecture does not require modifications for new sources, and the memory and compute overhead remains constant relative to standard transformers.
Experimental Assessment
Article-Level Unlearning in Wikipedia
NULLs is evaluated on a 1B-parameter model trained with 6 million Wikipedia articles as independent sources. Factual knowledge is categorized as either shared (present in multiple sources) or article-specific (unique to one article). Results show that, when a source's sink is disabled, the model's ability to recall unique article-specific facts collapses, while knowledge supported by other articles is retained identically to gold-standard retrained models.
Figure 2: NULLs selectively collapses recall on article-specific facts upon sink deactivation, while preserving shared information; post-hoc gradient-based unlearning indiscriminately degrades both.
Figure 3: NULLs' unlearning is virtually identical to retrained-from-scratch models—unique facts are reliably forgotten, while shared and inferred facts persist.
Topic-Level Unlearning: Harry Potter Case
NULLs supports coarse-grained (topic or cluster-based) unlearning. When trained on C4 and all Harry Potter books—with each as a separate cluster—suppressing the Harry Potter sink erases both verbatim and paraphrased knowledge of the books. Notably, triggering the sink toggles the model's ability to generate Potter-related content or entities, as demonstrated by prompt completions.
Figure 4: Disabling the Harry Potter sink elevates loss and eliminates Harry Potter knowledge, paralleling models never trained on the books; information is inaccessible regardless of paraphrase or entity-level prompt.
Adversarial Robustness
NULLs is shown to resist adversarial extraction and relearning attacks that compromise previous unlearning algorithms. Standard adversarial prompting (via GCG) fails to elicit unlearned content after sink suppression, with adversarial compression ratios (ACR) indistinguishable from retrained models.
Figure 5: NULLs matches retraining under adversarial extraction—targeted memorized content cannot be compressed out after unlearning, indicating robust removal.
Further, relabeling sinks and fine-tuning on unlearned data does not accelerate "relearning," in contrast to gradient-based post-hoc methods, where just a handful of iterations suffice to restore supposed forgotten information. NULLs instead mirrors the behavior of a clean retrained model under relearning attack scenarios.
Scaling and Knowledge Capacity
The capacity for memorization and unlearning is scalable. Increasing the sink pool size augments the amount of source-specific knowledge that can be encoded, while unlearning efficacy is only weakly affected by mask overlap until extreme cases. NULLs retains generalization and benchmark performance: on ARC-E, Winogrande, PIQA, and SciQ, the model is statistically indistinguishable from a standard transformer, demonstrating that fine-grained unlearning can coexist with high utility.
Figure 6: Increasing the sink pool—i.e., reducing overlap—boosts memorization capacity, yet Sink-Off (unlearned) performance remains close to retraining, signifying robust and scalable unlearning.
Practical and Theoretical Implications
NULLs refutes the prevailing assumption that source disentanglement and shared learning are fundamentally incompatible. By localizing source influence with deterministic, sparse masking, one can support retroactive compliance, attribution, or user-data deletion at arbitrary fine-grained levels. This design also yields a mechanism for direct measurement of source impact (by toggling mask activation), potentially supporting model auditing, data valuation, and provenance-tracking in future deployments.
The implementation is efficient: the deterministic mask generation allows for zero storage overhead for mask tables, and the computational penalty is negligible. NULLs can accommodate arbitrary definitions of "source," enabling adaptability to regulatory and application-specific requirements (e.g., GDPR "right to be forgotten," copyright requests, personalized models).
Limitations and Future Directions
This work is evaluated at the billion-parameter scale, and generalization to multi-billion or trillion-parameter models, as well as complex, multi-modal, or cross-source tasks, remains an open question. The approach presupposes that source definitions are available and meaningful relative to downstream unlearning requirements; unlearning queries that do not match pre-specified sources are not addressed. Extending NULLs to support arbitrary attributions post hoc, and to fine-tuning or continual training settings, is a compelling research avenue.
Conclusion
NULLs provides an efficient, scalable, and robust mechanism for source-controllable unlearning in LLMs without sacrificing generalization or downstream performance. By integrating source-specific sparse activation masks into transformer MLPs, NULLs achieves the practical utility of post-hoc unlearning methods with the precision and reliability of retraining, and exhibits strong resistance against adversarial and relearning attacks. This evidences that unlearnability can—and should—be designed as a native feature of LLM architectures, fundamentally advancing both data governance and AI compliance capabilities.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What is this paper about?
This paper is about teaching LLMs how to “unlearn” specific pieces of training data on command, without having to retrain the whole model. The authors introduce a new way to build and train LLMs—called NULLs (Natively Unlearnable LLMs)—so the model can keep general knowledge while cleanly removing anything learned from a particular source, like a single Wikipedia article or a whole book series.
What questions were the researchers trying to answer?
They focused on two simple questions:
- Can we build an LLM that learns general skills from many sources but keeps each source’s unique information separate, so it’s easy to remove later?
- If we “forget” a source, does the model still remember related facts it learned elsewhere, and does it stand up to tricks that try to bring the forgotten info back?
How does the new method (NULLs) work?
Think of the model’s brain like a school:
- The “backbone” is the shared school library: everyone uses it to learn general knowledge (like grammar or common facts).
- The “sinks” are small lockers assigned to each data source (like one locker for a specific Wikipedia article, or one for the Harry Potter books). These lockers are only opened when that source is being read and learned from.
When the model trains:
- Every document uses the shared library (the backbone) to learn general, widely useful patterns.
- Each source also activates its own tiny set of “locker” neurons (sinks) picked by a simple rule based on the source’s ID (like turning a source name into a consistent, random-looking key). Only that source uses that exact tiny set.
Why this helps:
- Facts that are unique to a source go into its locker because those locker neurons don’t get “interference” from other sources.
- Facts that many sources agree on end up in the shared library.
- To unlearn a source later, you just close that source’s locker. You don’t need to change any model weights or retrain the model.
In code terms, the model looks normal except for a small extra step inside its feed-forward layers (MLPs): after the usual computation, it multiplies the activations by a mask (on/off pattern) that turns on the shared neurons and the source’s locker neurons. Unlearning means not applying the locker mask for the source you want to forget.
What did they test and how?
They ran two main case studies to see if NULLs really works at both fine and broad levels.
1) Wikipedia: unlearning one article out of about 6 million
- They trained a 1B-parameter model on all of Wikipedia, treating each article as its own source.
- They built fill‑in‑the‑blank questions from articles and measured how strongly the model preferred the correct answer over good-sounding wrong answers (they call this a “truth ratio”).
- They compared three settings:
- Sink-On: the article’s locker is open (normal behavior).
- Sink-Off: the article’s locker is closed (simulating unlearning that article).
- Retrained: a gold-standard baseline where the whole model is retrained with that article removed.
- They also compared NULLs to common post-hoc unlearning methods (which try to erase data after training by tweaking weights).
What they looked for:
- Article-specific facts: details found only in that article.
- Shared facts: facts that appear in multiple articles (for example, general info about a city or a person that shows up in many places).
2) Harry Potter: unlearning a full topic at once
- They trained another model on general text plus all seven Harry Potter books, then gave the books their own “topic locker.”
- They checked:
- Loss (how well the model predicts the book text),
- Question-answer prompts about Harry Potter,
- Free-form generation (does it mention Hogwarts, etc.),
- Robustness tests:
- Adversarial prompting (crafty prompts to try to pull out “forgotten” text),
- Relearning attack (fine-tuning the model briefly to see if erased knowledge pops back quickly).
They also studied scaling and side effects
- They varied how many locker neurons a source gets and how much locker overlap there is between sources.
- They tested general language abilities on common benchmarks to make sure NULLs doesn’t reduce overall skill.
What did they find, and why does it matter?
Here are the main takeaways:
- Clean per-source unlearning that matches retraining: Closing an article’s locker (Sink-Off) makes the model forget facts unique to that article, but keeps shared facts intact. Its behavior is very close to the gold standard of full retraining without that article.
- Post-hoc methods blur and break things: Standard after-the-fact unlearning approaches not only remove the target info, they also damage nearby knowledge (for example, facts in related articles), and they can often be reversed.
- Robust to attacks: With Harry Potter, NULLs resists both adversarial prompts and quick relearning. When the locker is closed, it behaves just like a model that never saw the books, even under pressure.
- No loss in general skill: On standard language tasks, NULLs performs about the same as a regular transformer of the same size.
- Scales to millions of sources: Because many sources share the same pool of locker neurons (with small, unique masks), the model doesn’t blow up in size. You can control a huge number of sources without adding a separate module for each one.
This matters because it shows we don’t have to choose between learning well from lots of data and being able to cleanly remove specific pieces later. We can design unlearning into the model from the start.
What could this change in the real world?
- Better legal and privacy compliance: If a publisher or person asks to remove their content (for copyright or privacy reasons), the model can do it instantly by disabling the right lockers, without retraining.
- Clearer data responsibility: Because each source has its own locker, developers can more easily tell which sources influenced an answer and even measure the value of different data sources.
- Safer updates over time: Since unlearning is built in, models can adapt to takedown requests or policy changes without losing unrelated skills.
There are still open questions, like how to choose the best definition of a “source” (single documents, authors, topics, publishers), how to handle unlearning requests that don’t match those definitions, and how to preserve this unlearning ability during later fine-tuning stages. But overall, this work shows a practical path to making LLMs both powerful and controllable at the data level.
Knowledge Gaps
Below is a concise, actionable list of the paper’s unresolved knowledge gaps, limitations, and open questions for future work.
- Scaling to frontier models: The approach is only demonstrated on ~1B-parameter models; it is unknown how NULLs scales in performance, stability, and isolation quality on multi-10B+ and 100B+ LLMs, and how , , and should be tuned at scale.
- Capacity vs. number of sources: With millions of sources (e.g., 6M Wikipedia articles) and a fixed sink pool (e.g., , ), expected reuse per sink neuron is extremely high; a principled capacity analysis is missing to quantify collision rates, interference, and the point at which unlearning fidelity degrades.
- Formal guarantees of isolation: The paper shows empirical proximity to retraining but offers no formal guarantees (bounds) on leakage of source-specific information into backbone neurons as a function of overlap ratio, training steps, or corpus composition.
- Robustness breadth: Adversarial robustness is tested on one topic (Harry Potter), one attack family (GCG), and small-scale fine-tuning; resilience to broader attacks (e.g., weight inversion, membership inference, jailbreak variants, instruction-tuned red-teaming) and in larger models remains unassessed.
- Quantization and compression: Prior work shows post-hoc unlearning can be reversed by quantization; the effect of quantization, pruning, and distillation on NULLs’ isolation and unlearning guarantees is not evaluated.
- Permanent deletion side effects: “Zeroing out” sink parameters for a source may inadvertently remove information for other sources that overlap on those sink neurons; the paper does not quantify this collateral damage or propose safe per-source deletion when masks collide.
- Inference routing at scale: Practical mechanisms for selecting which mask(s) to apply at inference time are unclear—how to map arbitrary user queries to one or many sources, how many masks to compose, and how to avoid bias or retrieval errors in mask selection.
- True “Sink-Off” policy: Experiments often use “next-closest source” as Sink-Off; real unlearning requires disabling the target sink without rerouting to a near neighbor. The behavioral gap between “no sink at all” and “next-closest sink” is not measured.
- Multi-source composition: Many answers draw on multiple documents; the paper does not study combining multiple masks simultaneously, interference under mask unions, or training-time incentives for stable multi-mask composition.
- Requests misaligned with predefined sources: The framework assumes unlearning requests align with pre-declared sources; handling requests that cut across sources (e.g., an author, a time range, a concept, or an attribute like PII) is left open.
- Source granularity and hierarchy: How to define, learn, and support hierarchical/overlapping sources (document → publisher → topic) and enable unlearning at any level without retraining or reassigning masks is not explored.
- Dynamic/continual data: It is unclear how to add new sources post hoc (assign new masks) and fine-tune without degrading prior localization or re-entangling sources; strategies for continual learning with preservation of unlearnability are needed.
- Post-training compatibility: The paper flags but does not test whether SFT/RLHF/instruction tuning preserves source isolation, nor which regularizers or training protocols best maintain NULLs’ native unlearnability after alignment.
- Attention pathways: Only MLPs are masked; the role of attention layers (where knowledge can also reside) in leaking source-specific information is not measured; whether masking should extend to attention (e.g., value/MLP fusion) is open.
- Training stability and optimization: The impact of multiplicative sink masking on optimization dynamics (gradient sparsity, learning-rate schedules, initialization, layer-wise placement of sinks) is not systematically studied.
- Efficiency and systems overhead: The compute/memory/latency overhead of per-example mask generation, cross-document attention masking, and larger MLPs is not reported; deployment costs on real serving stacks (KV cache, batching) remain unknown.
- Cross-document packing and long contexts: While cross-document attention masking is used, the effect on long-context training, packed sequences, and multi-source windows in realistic pipelines (and any residual leakage) is not characterized.
- Evaluation breadth: General capability is tested on four small benchmarks; effects on instruction-following, reasoning, tool use, coding, safety alignment, multilingual understanding, and long-context tasks are unmeasured.
- Fact complexity: Truth Ratio probes mostly single-hop Cloze facts; the impact of unlearning on multi-hop reasoning, compositional facts, and counterfactual consistency is not evaluated.
- Measurement validity: Truth Ratio relies on GPT-generated paraphrases and negatives; potential bias, stability across paraphrasing, and correlation with human judgments or retrieval-augmented baselines are not validated.
- Attribution reliability: Although NULLs enables toggling sinks, it is unproven whether sink activation differences yield reliable source attribution/valuation across tasks; comparisons to principled data valuation (e.g., Shapley) are missing.
- Privacy guarantees: Beyond empirical robustness, there is no formal privacy or erasure guarantee (e.g., DP-style bounds, certification protocols) to support regulatory claims of deletion under GDPR/CCPA.
- Security of mask assignment: Deterministic PRNG-based masks raise questions about mask-key management, tamper resistance, and whether adversaries can infer or exploit mask structure to re-activate unlearned content.
- Multilingual and multimodal extension: Applicability to non-English corpora and to vision/language or audio/text models is untested; how to define and localize sources across modalities remains open.
- Bias and fairness: Unlearning certain sources could systematically shift model behavior; effects on bias/fairness metrics and methods to monitor/mitigate distributional drift from unlearning are not studied.
- Data curation at scale: Proposed uses for data valuation and redundancy removal are not demonstrated; protocols to iteratively prune/augment corpora using sink toggles (and avoid overfitting to sinks) need development.
- Allocation policies: Choosing adaptively per-source (size, importance, sensitivity) to handle long-tailed sources and capacity constraints is not addressed.
- Interaction with MoE and other architectures: How NULLs integrates with MoE routing, encoder-decoder models, retrieval-augmented systems, or KV-caching strategies, and whether hybrid designs improve isolation, is unexplored.
- Legal auditability: Practical procedures to audit and prove erasure (e.g., standardized tests, certificates, red-team protocols, logs) and to manage compliance lifecycles (unlearn → verify → monitor) are not specified.
Practical Applications
Immediate Applications
Below are actionable uses that can be deployed now, based on NULLs’ shared backbone + per‑source sink design and the paper’s empirical robustness to adversarial extraction and relearning.
- Copyright and licensing takedowns in production LLMs (media, software)
- Sector: media/publishing, software/dev tools
- What you can do: Train new models with per‑publisher, per‑document, or per‑repository source IDs. Upon a takedown request, disable the corresponding sink at inference (or permanently zero its sink weights) to approximate retraining without the source.
- Tools/products/workflows:
- Source registry and ID assignment in the data pipeline
- Deterministic mask generator library seeded by source ID
- “Unlearning controller” service that maintains a deny‑list of sinks to disable at inference
- Verification harness (truth‑ratio and adversarial‑prompt tests) to issue deletion reports
- Assumptions/dependencies: Requires training with NULLs from scratch and a clean mapping of data to sources; regulator/customer acceptance that sink‑off ≈ retraining; minimal routing logic to ensure masks are applied/disabled correctly.
- GDPR/CCPA Right‑to‑be‑Forgotten operations for customer or personal data (healthcare, finance, consumer tech)
- Sector: healthcare (HIPAA‑adjacent), finance, consumer apps
- What you can do: Assign per‑user or per‑institution sources during pretraining or post‑training on private corpora; on revocation, disable those sinks.
- Tools/products/workflows:
- Per‑tenant or per‑user source IDs in fine‑tuning/post‑training stages
- Audit logging that records sink states at query time
- Policy engine integrating privacy requests with sink toggles
- Assumptions/dependencies: Source granularity must align with anticipated unlearning requests (user/tenant/document); strong data provenance is required; privacy controls must extend to post‑training data if used.
- Enterprise data governance and model provenance auditing (all industries)
- Sector: enterprise AI/IT, data platforms
- What you can do: Attribute model behavior to training sources by comparing outputs with specific sinks on vs. off; identify low‑value or risky sources to prune.
- Tools/products/workflows:
- Attribution auditor: batch toggles of sinks and delta‑metrics
- Data valuation dashboard to score source impact and redundancy
- Assumptions/dependencies: Statistical attribution (truth‑ratio shifts) must be accepted as proxy; requires storage of sink–source mappings and evaluation compute.
- Rapid compliance geofencing and policy enforcement (government, regulated industries)
- Sector: public sector, legal/compliance, online platforms
- What you can do: Disable sinks tied to jurisdiction‑restricted content (e.g., region‑specific legal texts or sensitive material) for geofenced deployments.
- Tools/products/workflows:
- Policy‑aware routing that disables sinks based on user region or context
- Assumptions/dependencies: Accurate source labeling by jurisdiction; clear governance on what constitutes region‑restricted sources.
- License‑aware developer code models (software engineering)
- Sector: software
- What you can do: Tag training code by repository/license; disable sinks for codebases with license conflicts or opt‑outs without retraining the whole model.
- Tools/products/workflows:
- Repository‑level source IDs; CI/CD hooks to update license status and sink deny‑lists
- Assumptions/dependencies: Stable repo identifiers; overlap ratio and mask capacity sized for the number of repos.
- Per‑tenant/vertical LLMs with instant offboarding (SaaS/enterprise)
- Sector: enterprise SaaS
- What you can do: Allocate sinks per tenant or per dataset share; instantly remove a tenant’s data contribution by disabling their sinks when they churn or at contract end.
- Tools/products/workflows:
- Tenant provisioning that issues source IDs; offboarding playbook to disable sinks and generate compliance certificates
- Assumptions/dependencies: Clear separation of tenant data during training/post‑training; tenants must accept approximation to retraining.
- Safer dataset updates and recalls (all sectors)
- Sector: data platforms, MLOps
- What you can do: When a data source is later found harmful/poisoned, disable its sink immediately and schedule permanent zeroing during maintenance windows.
- Tools/products/workflows:
- Incident response runbook: sink disable → leakage check → optional parameter zeroing
- Assumptions/dependencies: Capability to quickly identify impacted source IDs; monitoring for leakage into backbone.
- Academic experimentation and controlled ablations (academia/R&D)
- Sector: academia, research labs
- What you can do: Run precise data ablations by toggling sinks for defined corpora/clusters; study generalization vs. memorization without repeated retraining.
- Tools/products/workflows:
- Experiment manager that turns sinks on/off and collects performance metrics
- Assumptions/dependencies: Well‑chosen source definitions (document‑, cluster‑, or domain‑level) to match study goals.
- Content moderation and category “kill‑switches” (online platforms)
- Sector: social platforms, content providers
- What you can do: Cluster and tag content categories (e.g., specific fandoms, copyrighted series, or known disinformation corpora) as sources; disable those sinks when policy changes.
- Tools/products/workflows:
- Semantic clustering pipeline; category sink toggles linked to policy flags
- Assumptions/dependencies: Category boundaries must be stable and operationalized as source IDs; monitoring for backbone leakage is advised.
- Model delivery with “unlearning SLAs” (AI vendors)
- Sector: AI model providers
- What you can do: Offer contractual SLAs to remove customer‑named training sources within hours by sink deactivation and verification—without retraining.
- Tools/products/workflows:
- Customer portal for unlearning requests; automated verification suite; certificate generation
- Assumptions/dependencies: Models must be trained with NULLs; legal teams must agree on verification standards and evidence thresholds.
Long‑Term Applications
These use cases require further research, scaling, or engineering (e.g., larger models, improved routing, finer‑grained sources, post‑training integration).
- Fine‑grained “per‑snippet” or “per‑entity” unlearning (document paragraphs, sentences, entities)
- Sector: media, legal, documentation systems
- Potential product/workflow: Granular source assignment (e.g., per‑passage hashing) with scalable sink pools; router that activates the most relevant sinks per query.
- Dependencies: Efficient routing to many small sources; stronger guarantees against leakage at extreme granularity; larger sink pools and overlap tuning.
- User‑controlled personal assistants with revocable on‑device memory
- Sector: consumer tech, mobile/edge AI
- Potential product/workflow: Each user/session becomes a source; users can clear their history by disabling or zeroing sinks locally.
- Dependencies: Compact on‑device NULLs variants; UI/UX for memory control; privacy‑preserving post‑training that maintains source localization.
- Federated and multi‑party training with robust per‑client sinks
- Sector: healthcare, finance, multi‑org consortia
- Potential product/workflow: Train shared backbone across clients while isolating client‑specific sinks; clients can demand deletion without retraining the federation.
- Dependencies: Secure aggregation with stable source IDs; protocols to prevent sink collisions and leakage; legal governance.
- Data licensing marketplaces with dynamic paywalls
- Sector: media, data brokers
- Potential product/workflow: Publishers license “data plug‑ins” whose sinks are activated only for paying deployments; automated royalty reporting via attribution deltas.
- Dependencies: Standardized source IDs across vendors; trusted accounting/audit mechanisms; interoperable mask APIs.
- Regulatory “provable unlearning” certifications
- Sector: regulators, auditors, legal tech
- Potential product/workflow: Standard test suites (truth‑ratio, adversarial compression ratio) tied to sink toggles; third‑party attestation.
- Dependencies: Accepted standards for evidence; legal acceptance that sink suppression approximates retraining; robust adversarial evaluations.
- Post‑training alignment that preserves source localization
- Sector: AI labs
- Potential product/workflow: RLHF/SLM post‑training with regularizers or constraints that maintain sink isolation; tools to detect/post‑training leakage.
- Dependencies: New algorithms and diagnostics to preserve NULLs’ disentanglement through instruction tuning and preference optimization.
- Multimodal extension (text–image–audio–video) with per‑source sinks
- Sector: creative industries, surveillance‑sensitive domains
- Potential product/workflow: Source‑aware masking across modality‑specific MLPs or adapters; per‑creator takedowns for images/audio/video.
- Dependencies: Architecture generalization beyond text MLPs; cross‑modal routing and consistent source IDs.
- Safety “circuit breakers” for emergent risky capabilities
- Sector: safety engineering, policy
- Potential product/workflow: Cluster training data that most contributes to unsafe behaviors; disable corresponding sinks as a rapid mitigation.
- Dependencies: Reliable source‑level attribution of capability emergence; low leakage into backbone; continuous monitoring.
- Regional and temporal policy toggles (geopolitical and time‑limited restrictions)
- Sector: global platforms, news/media
- Potential product/workflow: Enable/disable sources by region/time (e.g., embargoed content), with automated reactivation on expiry.
- Dependencies: Robust geo/time tagging of sources; policy engines integrated with inference‑time sink routing.
- Scientific reproducibility and causal attribution at scale
- Sector: academia, open science
- Potential product/workflow: Standardized evaluation that isolates the causal effect of datasets by sink toggling, facilitating transparent model cards.
- Dependencies: Community tooling and benchmarks; shared corpora with agreed source IDs; compute to run large ablations.
- Integration with Mixture‑of‑Experts and frontier models
- Sector: AI labs, hyperscalers
- Potential product/workflow: Hybrid MoE‑NULLs where experts share backbones and hold source‑specific sinks; unlearning at frontier scale.
- Dependencies: Engineering to maintain throughput/latency; careful overlap tuning and memory budgets; demonstration at >10B parameter scales.
- Continuous learning with reversible “feature patches”
- Sector: enterprise AI
- Potential product/workflow: Treat new updates as versioned sources with sinks that can be switched on/off for rollback and A/B testing.
- Dependencies: Deployment infra for versioned sink management; monitoring to detect regressions and leakage.
- Education and publishing with time‑boxed syllabus/content access
- Sector: education technology, publishers
- Potential product/workflow: Per‑course or per‑title sinks activated for enrolled learners; automatic sink deactivation after term or license lapse.
- Dependencies: Identity and entitlement management; courseware tagging; routing prompts to relevant course sinks.
Notes on feasibility and deployment
- Training requirement: NULLs must be adopted during training (or post‑training for the data it touches); existing non‑NULLs models cannot be retrofitted without retraining.
- Source definition is strategic: Choose source granularity (document, publisher, cluster, tenant, user) to match legal/commercial unlearning needs.
- Routing at inference: Real systems need a router to select applicable sinks for a query, or to enforce deny‑lists that disable prohibited sinks globally.
- Leakage monitoring: Although the paper shows strong robustness at 1B scale, larger models and different domains require monitoring for backbone leakage; overlap ratio and sink capacity must be tuned.
- Verification: Build acceptance criteria (e.g., truth‑ratio distributions, ACR) into compliance reports to demonstrate equivalence to retraining.
- Security and privacy: Protect the source–sink registry; ensure adversaries cannot infer disabled sources or reconstruct sensitive content via routing manipulation.
Glossary
- Adversarial Compression Ratio (ACR): A metric quantifying how much text can be elicited per unit-length adversarial prefix, used to measure latent memorization. "We report our results with the Adversarial Compression Ratio (ACR) \citep{schwarzschild2024rethinkingllmmemorizationlens}, which quantifies latent memorization as the ratio between the length of the reproduced text and that of the shortest adversarial prefix needed to elicit it."
- Adversarial Extraction: The recovery of targeted knowledge from a model via adversarial means after intended removal. "NULLs resists both adversarial extraction and relearning that reverses post-hoc unlearning."
- Adversarial Prompting: Crafting prompts specifically to elicit information that should be hidden or removed. "While suppressing the Harry Potter sink prevents the relevant knowledge from being elicited through standard prompts, it could still remain accessible via adversarial prompting \citep{patil2023sensitiveinformationdeletedllms}."
- Backbone (Shared Backbone Neurons): The shared neuron subset that learns general, cross-source capabilities. "by training a set of shared backbone neurons alongside a pool of sparsely activated sinks."
- Cloze-style Question: A fill-in-the-blank evaluation format to test factual recall. "We then use GPT-5 to convert each into a Cloze-style question paired with a set of plausible but incorrect answers."
- Cross-Document Attention Masking: An attention constraint to prevent information leakage across documents during training. "We also implement cross-document attention masking to prevent information leakage between sink activations when a training context contains text from multiple documents."
- Data Attribution: Tracing model outputs back to the data sources responsible for them. "data attribution \citep{li2023surveylargelanguagemodels} aims to trace the model's outputs back to responsible data sources."
- GCG Optimization: An adversarial optimization procedure used to find prompts that elicit targeted content. "we use GCG optimization \citep{zou2023universaltransferableadversarialattacks} to identify adversarial prompts that elicit unlearned text."
- Gold-Standard Retraining: Fully retraining a model on a dataset with the target source removed, used as the ideal baseline for unlearning. "We compare NULLs against the gold standard of retraining without the target source, across the unique, inferred, and shared facts defined above."
- Memorization Sinks: Architectural components (neuron pools) designed to localize and store source-specific or memorized information. "We implement NULLs based on the Memorization Sinks architecture introduced in \citet{ghosal2025memorizationsinksisolatingmemorization}."
- Mixture-of-Experts (MoE) Models: Architectures that route inputs to specialized expert modules, often per domain or source. "In mixture-of-experts models, \citet{shi2025flexolmo, gururangan2021demixlayersdisentanglingdomains} allocate separate expert modules to different data sources and domains."
- Natively Unlearnable LLMs (NULLs): Models designed so source-specific knowledge can be disabled without weight updates, preserving joint learning. "We propose NULLs (Natively Unlearnable LLMs), a model class that satisfies the two opposing goals of isolating source-specific contributions and learning jointly across sources, by training a set of shared backbone neurons alongside a pool of sparsely activated sinks."
- NPO: A gradient-based unlearning baseline method used for comparison in experiments. "Figure~\ref{fig:gradunlearning} compares two gradient-based unlearning baselines (NPO and gradient ascent) on source-level unlearning in Wikipedia."
- Overlap Ratio: The expected fraction of shared sink neurons between different sources’ masks. "We vary the overlap ratio at a fixed $N_{\textrm{source}$ and compare Sink-On (ground-truth article sink active) to Sink-Off (next-closest article sink active) and Retrained."
- Parameter Editing: Directly modifying selected model parameters to change specific behaviors or knowledge. "using either gradient-based tuning or parameter editing."
- Post-hoc Unlearning: Removing information from a trained model via updates after training completes. "Post-hoc methods modify a fully trained model to remove targeted information after training."
- Provenance: The origin or source of data used to train a model. "A source may be a unit of provenance, such as a document, a publisher, or a cluster of topically related documents."
- Pseudo-Random Number Generator: A deterministic generator used to create per-source masks reproducibly from source IDs. "We create the mask with a pseudo-random number generator, allowing it to be generated on the fly during training or inference."
- Relearning (via Fine-tuning): The process by which previously unlearned information is re-acquired through additional fine-tuning. "Relearning via Finetuning"
- Semantic Deduplication: Identifying semantically equivalent or highly similar content to avoid duplicates across sources. "We identify whether a fact appears across multiple articles with semantic deduplication \citep{minishlab2025semhash}."
- Sink Neurons (Sink Pool): A dedicated pool of neurons selectively activated per source to localize source-specific information. "we activate a subset of size $N_{\textrm{source}$ of the $N_{\textrm{pool}$ sink neuron pool while dropping out the remainder."
- Sink-On / Sink-Off: Inference modes toggling the activation of the target source’s sink versus a non-target sink. "Throughout our experiments, we evaluate two inference modes: Sink-On, in which the ground-truth source sink is activated, and Sink-Off, in which the next-closest source (by embedding similarity) is activated instead."
- Source-Dependent Mask: A deterministic, per-source sparsity mask applied to sink neurons during activation. "The post-nonlinearity activations are multiplied by a source-dependent mask which activates all shared backbone neurons but only a consistent fraction of the sink neuron pool."
- Source-Level Unlearning: Removing the influence of a particular data source while preserving shared knowledge. "Together, these results suggest that source-level unlearning need not be an afterthought."
- Truth Ratio (TR): A likelihood ratio comparing the correct answer against plausible distractors to assess factual recall. "We measure the model's knowledge of a fact via the Truth Ratio (TR), the ratio of the likelihood of the correct answer to that of a set of plausible but incorrect answers:"
Collections
Sign up for free to add this paper to one or more collections.