Distribution-free conformal transfer under arbitrary attacker and release drift

Establish distribution-free transfer guarantees for conformal methods under arbitrary prompt, model, or external-knowledge-snapshot drift affecting the attacker or release configuration.

Background

Conformal Privacy Auditing (CPA) calibrates certificates for a declared attacker and release configuration under exchangeability. The paper’s stress tests show that certificates calibrated under one configuration can lose coverage when evaluated under a different release mechanism or attacker-side configuration, such as changing the attacker’s profile database. Recalibration is therefore required operationally.

The paper notes that variants such as Mondrian or weighted conformal prediction may address explicitly modeled drift, but they require additional assumptions. The unresolved problem is to obtain distribution-free transfer guarantees when drift is arbitrary, including changes to prompts, models, or external web snapshots used by the attacker.

References

A CPA certificate is thus calibrated for a declared threat configuration and does not automatically transfer under attacker or release drift---recalibration is the operational protocol. More robust variants (e.g., Mondrian or weighted conformal prediction) are possible when the drift model is explicit, but do not provide distribution-free transfer under arbitrary prompt, model, or snapshot drift, which remains an open problem for conformal methods.

— Conformal Privacy Auditing: Calibrated Re-identification Attacks with Statistical Guarantees  (2609.21340 - Huang et al., 18 Sep 2026) in Section 6, subsection “Robustness under Stressed Assumptions,” paragraph “Exchangeability and attacker/release drift”